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Preface 


This  study  sought  to  design  a  more  efficient  access 
method  to  the  appointment  system.  Using  the  systematic 
approach  of  the  design  methodology,  a  single  'best'  solution 
was  derived.  Implementation  of  the  proposed  system  will 
increase  the  availability  of  the  appointment  system  to  the 
patient,  the  true  customer. 
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suggestions.  Thanks  goes  to  Major  Don  Shields  of  the 
Medical  Center's  office  of  Medical  Computer  Systems,  whose 
concerns  became  this  thesis  and  whose  support  provided 
timely  inside  information.  I  also  wish  to  thank  my  wonder¬ 
ful  wife  Sandy  whose  encouragement,  patience,  and  pioneering 
spirit  helped  me  realize  I  could  complete  this  work.  I 
appreciate  the  forgiving  nature  of  my  children,  Addie  and 
Jonathan,  who  couldn’t  understand  what  Daddy  was  alwjiys 
doing,  but  knew  he  still  loved  them.  Finally,  I  give  thanks 
to  the  Creator  who  reminded  me  repeatedly  that  'all  things 
are  possible  through  Him  who  strengthens  me.' 
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Abstract 

The  purpose  of  this  study  was  to  select  an  optimal 
candidate  system  that  would  meet  the  needs  identified.  In 
the  process  the  Justification  for  selection  was  developed. 

The  study  first  conducted  an  indepth  needs  analysis  of 
the  problem,  which  was  defined  as  the  poor  accessibility  to 
the  Medical  Center  appointment  desk.  Secondly,  a 
feasibility  study  was  completed  that  captured  problem  data 
and  organized  into  a  more  meaningful  analysis  of  the 
problem.  Finally  a  preliminary  activities  phase  established 
quantifiable  criteria  to  assist  in  the  selection  of  an 
optimal  candidate  system. 

The  methodology  established  17  measurable  parameters  to 
quantify  the  criteria  chosen.  This  quantification  process 
assisted  in  the  development  of  a  single  criterion  function. 
The  utilization  of  this  function  allowed  for  the  selection 
of  a  single  candidate  system  based  on  a  cardinal  scale. 
Sensitivity  of  the  total  criterion  function  value  to  each  of 
the  parameters  was  identified.  Any  changes  to  the  chosen 
candidate  system  parameters  should  avoid  affecting  those 
parameters  identified  as  having  the  greatest  sensitivity. 

The  candidate  system  chosen  relied  on  the  use  of  an 
automated  voice  messaging  system.  A  review  of  voice 
messaging  technology  showed  the  clear  advantage  this 
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communications  application  has.  While  no  product  endorse¬ 
ments  were  made,  or  suggested,  Microlog  Corporation’s  VCS 
3000  automated  voice  messaging  system  was  reviewed  as  an 
illustration  of  voice  messaging  equipment. 

Recommendations  Included  the  need -for  further  research 
that  would  improve  other  aspects  of  the  appointment  system. 
Duplication  and  verification  of  the  study  itself  may  improve 
the  criteria  selected  and  the  choice  of  measurable  para¬ 
meters.  Additional  areas  of  research  may  also  be  directed 
toward  the  evaluation  of  available  commercial  voice 
messaging  systems.  Implementation  of  any  particular  system 
was  not  addressed  but  was  assumed  to  a  be  a  matter  more 
appropriately  left  to  the  local  commander’s  discretion. 


IMPROVING  ACCESS  TO  THE  APPOINTMENT  SYSTEM  AT  THE 


USAF  MEDICAL  CENTER  WRIGHT-PATTERSON  AFB  OH 


I .  INTRODUCTION 


General  Issues 

According  to  James  Quinn,  in  an  article  for  Scienti fie 
America .  "efficient  and  high-quality  services  are  crucial" 
to  consumers  (16:50).  The  services  required  of  medical 
appointment  scheduling  rely  on  an  effective  system  that 
stresses  efficiency  and  quality  goals.  With  the  heavy  daily 
demand  for  appointments,  these  goals  can  become  a  particular 
challenge  for  hospital  appointment  systems. 

An  effective  appointment  system  will  satisfy  the  needs 
of  both  staff  and  patients.  To  be  efficient,  the  system 
should  minimize  the  resources  required  to  achieve  optimal 
results.  Various  system  configurations  have  been  attempted 
in  an  effort  to  make  appointment  systems  more  effective  and 
ef  f icient . 

Traditionally,  appointments  were  scheduled  using  manual 
bookkeeping  methods.  For  some  situations  this  manual 
scheduling  of  appointments  continues  today  but  depending  on 
patient  volume,  remains  a  tedious  and  time-consuming  task. 
Automated  systems,  however,  have  been  developed  utilizing 
the  computer’s  ability  to  efficiently  handle  routine  and 
repetitive  tasks.  Although  computers  have  simplified  much 
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of  the  bookkeeping,  the  connection  between  the  ouetomer  and 
the  appointment  system  remains  a  human  operator.  It  is 
through  this  mediator  that  patients  gain  access  to  the 
appointment  desk  and  can  then  realize  the  benefits  of 
computer  efficiency. 

As  part  of  a  hospital’s  overall  goal,  an  effective 
appointment  system  can  add  substantial ly  to  a  patient’s 
satisfaction.  When  considering  the  need  for  an  effective 
and  efficient  appointment  system,  one  might  consider  the 
following  French  proverb:  "While  we  keep  a  man  waiting,  he 
reflects  on  our  shortcomings"  (5:421). 

Sped  tic  Problem 

Patients  seeking  a  routine  or  follow  up  appointment  at 
the  USAF  Medical  Center,  Wr ight-Patterson  AFB  OH,  must  first 
contact  the  appropriate  appointment  desk  either  in  person  or 
by  telephone.  The  Medical  Center  currently  makes  use  of 
a  de-central ized  appointment  process  that  delegates  the 
responsibility  of  scheduling  appointments  to  each  individual 
department.  The  appointment  system  relies  on  a  centralized 
appointment  scheduling  system  that  automates  most  of  the 
bookkeeping  efforts  and  integrates  all  subscribing 
departments  with  a  single  appointment  scheduling  software 
package  and  patient  database. 

If  a  patient  calls  an  appointment  desk  and  a  connection 
is  established,  the  call  is  either  answered  by  an  appoint- 
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ment  clerk  or  put  into  a  holding  queue.  Callers  reaching  a 
busy  signal  may  need  to  re-dial  Several  times  until  they  are 

able  to  get  through. 

Several  problems  exist  with  such  a  system.  The  most 
apparent,  are  the  inconvenience  associated  with  having  to 
re-dial  a  telephone  number  and  the  frustration  encountered 
when  placed  on  hold  with  no  indication  of  duration.  Major 
Donald  Shields,  Associate  Administrator  of  Resources  at  the 
Medical  Center,  feels  patient  satisfaction  could  be  greatly 
improved  by  reducing  the  amount  of  time  a  call  stays  in  this 
waiting  queue  (22) .  Another  problem  is  encountered  when  a 
call  is  connected  and  put  on  hold,  no  identification  is 
given  as  to  which  appointment  desk  has  been  reached  until 
the  call  is  actually  answered,  the  one  exception  being  the 
pediatric  clinic. 

Finally,  a  significant  problem  exists  with  the  absence 
a  mechanism  to  prioritize  calls.  This  means  routine 
appointment  needs,  acute  care  appointment  needs,  and 
information  only  calls  are  mixed  together  and  responded  to 
in  a  f irst- in- f irst-out  (FIFO)  manner.  These  problems  all 
point  to  the  need  for  a  more  efficient  method  of  accessing 
the  appointment  scheduling  desk. 

Research  Ob j ectives 

Based  on  the  research  design  methodology  developed  by 
Benjamin  Ostrofsky,  University  of  Houston  TX,  a  detailed 
examination  of  the  problem  was  performed.  The  first  three 
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steps  in  the  methodology;  Needs  Analysis,  Feasibility 
Study,  and  Preliminary  Activities,  were  completed. 

The  first  step,  Needs  Analysis,  developed  a  statement 
of  the  specific  problem  based  on  the  existing  needs.  The 
objective  was  to  establish  the  need  for  the  project  and 
justify  further  research.  Step  two,  the  Feasibility  Study, 
'developed  a  set  of  useful  solutions  to  meet  the  needs'  of 
the  existing  problem  (14:29).  This  set  of  solutions  was  the 
result  of  an  exhaustive  review  of  potential  methods  that  are 
available  to  satisfy  the  stated  needs.  Initially,  no  system 
was  excluded  simply  because  of  operability  or  feasibility. 
The  objective  of  the  Feasibility  Study  was  to  acquire  all 
possible  ideas  so  as  to  eventually  capture  the  one  best 
candidate  system  that  satisfies  the  needs  of  the  original 
problem. 

Finally,  having  developed  a  set  of  useful  solutions, 
the  last  step  addressed  the  Preliminary  Activities.  This 
analysis  identified  the  best  candidate  system  from  the 
set  of  solutions  originally  considered.  While  the  most 
theoretically  favorable  or  optimum  candidate  system  is 
desired,  it  may  be  necessary  to  settle  for  the  optimal 
candidate  system  'which  is  the  most  favorable  for  the 
criteria  and  the  set  of  candidates  defined'  (14:71). 
Ostrofsky  advocates  the  formalism  of  the  methodology  as  an 
objective  approach  which  should  result  in  a  better 
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performing  candidate  system  than  what  might  ‘emerge  from 
some  other,  less  objective  approach'  (14:72). 

While  these  activities  may  seem  iterative  in  nature 
they  lead  up  to  the  final  objective  which  is  the  selection 
of  the  best  possible  solution  to  the  problem  identified. 
Obviously  this  ‘best’  solution  is  dependant  upon  the 
information  available  and  decisions  made  in  the  process  of 
development.  The  resulting  solution  will  meet  the  needs 
identified  and  be  an  effective  choice  for  implementation. 

Research  Scope 

The  appointment  system  is  a  complex,  interactive 
sequence  of  events.  Within  this  flow  of  events  are  many 
subsystems,  each  having  their  own  unique  requirements.  In  a 
general  sense,  Brandler  divides  the  appointment  system  into 
‘appointment  desk  operations  and  appointment  availability, 
[which]  can  be  viewed  as  separate  items  for  evaluation 
purposes'  (3:25).  As  an  existing  subsystem,  the  problem 
with  access  to  the  appointment  desk  was  further  analyzed. 

In  accordance  with  the  prescribed  methodology,  a  few  of  the 
suggested  solutions  may  appear  outlandish,  but  the  likeli¬ 
hood  of  Including  the  optimum  candidate  system  increases 
with  the  inclusion  of  all  possible  solutions. 

The  USAF  Medical  Center  at  Wright-Patterson  AFB  is  a 
large  complex  with  a  variety  of  departments  and  scheduling 
needs.  Currently,  the  following  departments  fall  under 
hospital  services: 
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1 .  Dental  Services 

2.  Environmental  Health  Service 

3.  Internal  Medicine,  Including  Neurology 

4.  Mental  Health 

5.  Obstetrics  and  Gynecology 

6.  Orthopedics 

7.  Pediatrics 

8.  Primary  or  Acute  Care 

9.  Radiology 

10.  Surgery 

With  the  exception  of  Mental  Health,  Radiology,  and  Environ¬ 
mental  Health  Services,  the  remaining  seven  departments 
employ  an  automated  appointment  system  that  utilizes  a 
centralized  patient  database.  This  study  was  concerned  with 
patient  access  to  the  automated  appointment  system. 

The  final  section  in  Ostrofsky’s  methodology  delineates 
the  processes  required  to  implement  the  chosen  candidate 
syst-em.  The  necessary  activities  needed  in  the  production- 
consumption  cycle  are  explained  and  lead  to  the  actual 
implementation  of  the  proposed  system  into  a  production 
process.  As  this  research  is  only  a  study  in  system 
feasibility,  the  decision  to  fund  actual  implementation  must 
rest  with  the  proper  authorities  and  is  beyond  the  scope  of 
this  research. 

Conclusion 

Inadequate  access  to  the  hospital  appointment  system  is 
the  result  of  several  factors.  The  implementation  of  a 
formalized  methodology  is  used  to  analyze  the  stated  problem 
and  provide  a  single  'best'  solution.  The  following  review 
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of  literature  will  examine  the  difficulties  others  have 
encountered  with  appointment  systems. 
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Literature  Review 


il- 

Generally,  appointment  systems  are  an  attempt  to 
provide  fair  and  responsive  treatment  to  all  patients. 
Increasing  patient  workloads  have  tested  both  the  efficiency 
and  effectiveness  of  traditional  appointment  systems. 

Appointment  Systems 

When  Casper  Weinberger  became  the  Secretary  of  Defense 
in  1981,  he  established  as  prime  goals  of  the  administra¬ 
tion:  the  'elimination  of  duplication  and  waste’ 

(6;Chap  13,19).  This  top  management  concern,  reinforced  by 
Congressional  demands,  has  forced  defense  agencies  to 
carefully  evaluate  their  spending.  One  agency,  the  USAF 
Medical  Service,  manages  a  large  and  complex  budget. 
Effective  management  of  the  Medical  Service’s  budget,  may  at 
times  require  a  reduction  in  resources,  forcing  managers  to 
be  efficient  in  as  many  areas  as  possible.  Such  constraints 
on  appointment  system  resources,  encourages  development  of 
more  efficient  procedures  in  this  arena. 

Contacting  an  appointment  desk  is  usually  a  patient’s 
first  step  in  obtaining  medical  treatment  (3:24). 
Consequently,  the  overall  perception  of  a  hospital  may  be 
based  on  the  efficiency  of  its  appointment  system.  For  both 
the  hospital  and  patients,  the  effects  of  a  good  appointment 
system  will  'optimize  the  available  resources  and  minimize 
the  waiting  time  of  patients'  (10:46).  From  a  patient’s 
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perspective,  making  an  appointment  should  be  relatively  easy 
and  convenient  (9:65). 

To  achieve  such  perceptions  an  appointment  system  must 
do  more  than  just  transfer  patient  appointment  information 
to  respective  departments  and  clinics.  A  successful 
appointment  system  will  involve  physician  and  patient 
punctuality,  management  control,  and  good  clerical 

procedures  (10:40). 

Problems  with  punctuality  may  indicate  an  imbalance 
with  a  hospital's  resources.  In  a  consulting  study  for  the 
U.S.  Public  Health  Service  Hospital  at  New  Orleans  LA,  Dr. 
Edward  Madden  Jr.,  Chief  of  the  Health  Service  Research 
Department,  discovered  that  it  is  more  often  the  physicians 
who  are  late,  versus  the  generally  assumed  patient  tardiness 
(10:49).  Such  imbalanced  management  can  detract  from  a 
patient's  overall  perception  of  a  facility.  Franz  Herpok 
believes  that  health  service  administrators  can  optimize 
'the  relationship  among  the  priorities  of  patients, 
providers  and  support  personnel'  through  a  balanced 
allocation  of  resources  (personnel,  funds,  physical  space) 
(9:66).  Such  an  endeavor  is  particularly  challenging  in  a 
military  environment  where  three  patient  populations  exist; 
active  duty,  dependent,  and  retiree. 

Effective  appointment  systems  are  the  result  of 
management  effort  and  are  of  great  importance  to  multi¬ 
faceted  care  providers  such  as  the  military  (26:392).  Major 
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Brandler,  in  an  article  for  the  USAF  Medical  Service  Digest . 
outlined  many  of  the  current  problems  associated  with 
military  medical  care  including  the  difficulties  with  the 
appointment  system.  Brandler  denoted  appointment  calls 
reaching  busy  signals,  being  placed  on  hold,  or  told  to  call 
back  later  as  a  significant  problem  (3:26).  He  further  went 
on  to  suggest  the  following  possible  solutions  for  improving 
access  to  the  appointment  system. 

1.  Information  brochure  explaining  scheduling 
process  and  times. 

2.  Stagger  scheduling  for  clinics. 

3.  Change  staffing  --  full-time  to  part-time, 
stagger  hours,  etc. 

4.  Add  equipment/staff,  if  necessary. 

5.  Automated  scheduling  system.  (3:26) 

Implementation  of  any  or  all  of  these  suggestions  depends  on 
effective  staff  coordination  and  department  organization 
(3:29)  . 

A  successful  system  is  based  in  part  on  organizing 
departments  in  the  most  efficient  manner.  This  may  include 
de-central i zed ,  centralized,  or  a  combination  of  both.  To 
better  understand  the  features  of  centrally  organizing 
appointment  scheduling  efforts,  Colonel  Richard  Stuart 
(USA),  Commander  of  the  U.S.  Army  Hospital  at  Bremerhaven, 
West  Germany,  surveyed  four  large  US  Army  hospitals  in  1972. 
From  this  study  he  drew  some  important  conclusions  that  can 
also  be  applicable  to  de-central ized  appointment  systems. 
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One  of  Stuart’s  major  findings  was  that  the  key  to  any 
good  appointment  system  is  an  efficient  telecommunications 
operation.  Without  adequate  patient  access,  even  the  most 
advanced  system  is  useless.  Based  on  his  study  of  military 
appointment  systems,  he  recommended  the  following; 

1.  A  need  for  adequate  and  dependable  staffing. 

2.  Limit  busy  signals  to  less  than  six  percent  of 
all  calls. 

3.  Appointments  should  be  made  only  by  telephone. 

4.  At  least  70  percent  of  outpatient  visits  should 
be  booked  in  advanced. 

5.  Hospitals  should  have  separate  numbers  for 
cancellations  and  for  long-distance  calls. 

(26:393) 

Additionally,  Colonel  Stuart  found  the  average  appointment 
call  lasted  between  2.1  and  2.4  minutes  and  information 
calls  averaging  less  than  2  minutes  (26:394). 

Stuart’s  research  established  many  standards  for 
military  hospitals  to  emulate.  His  concerns  for  efficient 
appointment  systems  in  military  medical  facilities  continue 
to  the  present.  This  is  evidenced  by  the  fact  that 
appointment  systems  were  the  number  one  item  on  the  USAF 
Surgeon  General’s  Information  Systems  top  ten  problem  list 
for  fiscal  year  1988  (27). 

Regardless  of  how  good  an  appointment  system  is, 
offering  a  wide  range  of  services  to  a  growing  number  of 
outpatients,  taxes  even  the  very  best  systems.  An 
overriding  factor  for  the  success  of  any  appointment  system 
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will  always  be  in  terms  of  patient  accessibility.  This 
access  is  constrained  by  the  numerous  variables  that  exist 
within  a  matrix  of  operations.  Through  a  careful  methodo¬ 
logical  analysis  of  multiple  concepts,  a  successful  system 
can  be  developed  that  will  satisfy  the  needs  of  the  problem. 

Methodology  Background 

Prior  to  the  twentieth  century,  development  of  products 
and  technologies  underwent  a  slow,  evolutionary  process 
(1:2).  During  the  1930’s  and  1940's,  the  complexity  of 
technology  quickly  widened  the  gap  between  the  initial  idea 
for  a  process  and  its  actual  utilization.  Hall  defined  this 
gap  as  'organized  creative  technology'  and  what  'shortens 
the'  gap  between  discoveries  and  their  subsequent  applica¬ 
tion'  is  what  he  called  'system  engineering'  (7:3) .  The 
increased  complexity  as  Hall  and  others  have  understood  it, 
has  brought  about  an  Increasing  number  of  interactions 
between  the  multiple  facets  of  manufacturing.  These 
interactions  involve  many  technical,  scientific,  and 
communication  problems  involving  individuals  with  varying 
backgrounds,  levels  of  ability,  and  skills  (14:3).  Some  of 
the  early  users  of  systems  engineering  included  RCA  during 
its  development  of  broadcast  television  service  in  the 
1930’s.  The  RAND  Corporation  developed  the  idea  of  systems 
analysis  or  initial  research,  which  became  the  foundation 
for  systems  engineering.  But  it  was  Bell  Laboratories  that 
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was  the  first  to  use  the  term  'system  engineering'  during 
the  early  1940’s  (7:7). 

The  objectives  of  systems  engineering,  included  the 
need  to  'balance  the  overall  development  program,  making  the 
best  use  of  manpower  and  resources'  (7:12).  This  efficiency 
would  be  the  result  of  careful  planning,  making  use  of  new 
knowledge,  and  providing  effective  communication  to  manage¬ 
ment  for  control  and  guidance  (7:12).  Understandably, 
system  engineering  was  not  without  its  drawbacks.  A  chief 
disadvantage  was  the  use  of  value  judgments  in  the  objective 
statement,  and  later  in  defining  which  objectives  in  the 
design  should  be  used  first  (7:11). 

In  1962  Morris  Asimow,  published  a  book  entitled 
Introduction  to  Design .  In  it,  Asimow  delineated  a  design 
morphology  that  answered  the  needs  for  system  engineering  in 
product  design.  He  realized,  as  others  before  him  had,  that 
in  today’s  society  there  is  a  shorter  amount  of  time  for 
development  of  increasingly  complex  designs  which  increase 
the  risk  of  error  or  failure  (1:2). 

Asimow  realized  that  like  the  different  approaches  in 
philosophy  there  were  also  many  ways  to  attack  a  problem  or 
set  of  problems  with  differing  views.  Towards  this  end  he 
outlined  a  fourteen  step  process  that  led  to  the  production 
phase  of  a  product  from  its  initial  needs  analysis.  The 
first  step  in  the  process,  dictated  that  the  design  must  be 
in  response  to  some  need,  either  social  or  individual,  that 
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can  be  met  by  technological  design  (1:5).  The  remaining 
outline  further  examines  the  need  for  the  project  in  terms 
of  physical,  financial,  and  design  constraints. 

It  was  quickly  seen  that  the  design  morphology  Asimow 
had  outlined,  provided  a  systematic  approach  to  careful 
planing  of  complex  systems.  Asimow  suggested  some  questions 
to  reflect  on  when  pondering  the  use  of  such  a  morphology. 

1.  When  designing  a  system  involving  thousands  of 
attributes,  which  ones  are  really  important? 

2.  From  who's  point  of  view  should  the  advantage  of 
an  attribute  be  ascribed? 

3.  How  much  confidence  can  we  attach  to  a  design 

to  be  carried  out  to  a  physical  product?  (1:24) 

The  result  of  Asimow’ s  work  was  an  orderly  sequence  of 
decisions,  which  when  resolved  emerged  with  an  effective 
answer  to  the  needs  identified  (14:3). 

While  the  work  of  both  Hall  and  Asimow  helped  the 
development  of  large  scale  planning  projects,  no  effort  was 
made  to  collectively  assemble  together  the  various 
philosophies  required  for  such  efforts.  Thus  the  need  for  a 
sequential  course  in  design,  planning,  and  decision  making 
was  born. 

Recognizing  this  need,  resulted  in  a  collaboration 
effort,  over  the  years,  between  Benjamin  Ostrofsky  and  his 
colleagues.  The  results  of  their  work  were  based  largely  on 
Asimow’ s  early  work  but  organized  into  a  less  technical  and 
more  manageable  methodology  for  Instructional  purposes. 
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Combining  both  old  and  new  concepts,  Ostrofsky  and  others 

allowed  managers,  designers  and  students  as  well  to; 

relate  logically,  some  for  the  first  time,  the 
requirements  of  their  activity  and  to  decide  for 
themselves  the  level  to  which  their  knowledge  and 
resources  would  engender  confidence  in  their 
conclusions.  (14:x) 

The  result  of  his  work  gave  the  user  a  better  comprehension 
of  real  world  problem  needs,  decisions  required  to  overcome 
them,  and  the  risks  associated  with  such  decisions.  Perhaps 
the  most  endearing  feature  of  Ostrofsky’s  methodology  is 
that  it  allows  a  student  to  accomplish  the  procedures  with 
his  own  abilities,  giving  him  confidence  and  success  with 
its  features.  It  is  important  to  point  out,  though,  that 
adhering  to  the  design  morphology  itself  does  not  guarantee 
a  successful  solution.  Making  such  a  claim  would  propose 
that  *any  problem  can  be  successfully  solved,  or  any  system 
successfully  designed,  if  one  were  only  to  follow  the  logic 
of  the  morphology"  (14:4).  The  methodology,  therefore,  is 
not  so  much  a  problem  solver  but  rather  a  structured  problem 
analyzer . 

Conclusion 

The  struggle  with  appointment  systems  has  grown 
through  the  years  with  increasing  patient  populations. 
Appointment  systems  for  military  medical  facilities  are 
particularly  challenging,  because  they  have  a  complex 
diversity  of  patients.  Studies  have  been  accomplished  that 
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point  to  the  need  for  a  more  efficient  and  effective 
appointment  system. 

To  accomplish  a  sufficient  study,  a  systematic  approach 
is  needed  to  analyze  the  problem.  Early  work  by  Arthur  Hall 
and  Morris  Asimow,  delineated  an  exhaustive  and  detailed 
approach  to  the  planning,  design,  and  development  of  complex 
engineering  problems.  Benjamin  Ostrofsky  expanded  on  these 
concepts  to  provide  a  design  methodology  teachable  at  the 
university  level.  The  chapters  that  follow  are  the  results 
of  implementing  his  methodology  and  examining  the 
appointment  system  accessibility  problem  and  its  needs. 


16 


Ill . 


Having  established  a  definitive  problem  of  accessing 
the  appointment  system,  this  chapter  is  devoted  to 
developing  ‘a  set  of  useful  solutions  to  meet  the  needs’  of 
that  problem  (14:29).  The  result  is  a  set  of  possible 
solutions  or  candidate  systems,  that  satisfy  the  problem. 

Needs  Analysis 

Having  defined  the  specific  problem  earlier  in  Chapter 
1,  a  needs  analysis  provides  the  justification  for 
additional  expense  and  effort  (14:31).  This  analysis  can  be 
done  in  a  variety  of  ways.  Ostrofsky  advocates  the  use  of 
surveys,  suggesting 

considerable  insight  can  be  gained  concerning  the 
characteristics  associated  with  the  consumer’s  or 
operator’s  ideas  about  the  nature  of  the  problem 
needs .  (14:33) 

After  consulting  with  Major  Shields,  it  was  agreed  that  a 
survey  would  be  more  beneficial  if  administered  to  staff 
members  who  routinely  work  with  the  appointment  system  or 
rely  on  its  information.  Furthermore  the  survey  was 
restricted  to  the  seven  departments  that  utilize  the  current 
automated  appointment  system  (22) . 

The  resulting  population  consisted  of  ten  officers  in 
executive  management;  seven  hospital  department  chairmen; 
and  eleven  appointment  clerks  and  their  seven  respective 
supervisors,  for  a  total  of  34  individuals  (35  positions 
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were  identified  but  one  officer  in  executive  management  also 


served  as  a  department  chairman).  The  questionnaire  Itself 
is  contained  in  Appendix  A.  A  stratified  tabulation  of 
results  can  be  found  in  Appendix  B. 

Questionnaire  results .  The  questionnaire  suggested  a 
generally  satisfied  feeling  with  the  present  appointment 
system  but  indicated  improvements  could  be  made.  For 
example,  in  response  to  question  six  Illustrated  in  Table  1, 
30  percent  of  the  population  was  either  dissatisfied  or  very 
dissatisfied  with  the  appointment  system  overall. 
Satisfaction  with  the  appointment  system  accounted  for  44 
percent  of  the  staff  with  Just  3  percent  being  very 
satisfied.  The  remaining  23  percent  were  neither  satisfied 
nor.  dlssatisf led  or  the  question  was  not  applicable  to  them 
based  on  their  job  description. 


Table  1.  Tabulated  Responses  to  Question  Six 


6.  What  is  your  overall 

opinion 

of 

the  appointment 

system  used  to  make 

appointments? 

N/A 

V/D 

D 

N/S/D 

S 

V/S 

0 

1 

2 

3 

4 

5 

Executive  Management 

33*4 

1  ir. 

22r. 

or. 

33% 

or. 

Department  Chair 

or. 

or 

43% 

43*4 

14% 

0% 

Subtotal 

19*4 

67, 

31*4 

19% 

25% 

0% 

Supervisor 

0*4 

or. 

29% 

14% 

57% 

or. 

Appointment  Clerk 

or. 

9r. 

9% 

9% 

64% 

9% 

Subtotal 

0*4 

67, 

17% 

11% 

61% 

6% 

COMBINED  TOTAL 

9*4 

67, 

24% 

15% 

44% 

3% 
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A  second  example  summarized  in  Table  2,  showed  24 
percent  of  the  population  being  very  dissatisfied  and  21 
percent  being  dissatisfied  with  question  five,  pertaining  to 
the  number  of  appointment  scheduling  people  in  each  clinic. 
Alternatively,  26  percent  were  satisfied  and  12  percent  were 
very  satisfied,  resulting  in  a  divided  opinion  on  the 
question.  The  remaining  17  percent  were  neither  satisfied 
nor  dissatisfied  or  the  question  was  not  applicable  to  them 
based  on  their  job  description. 


Table  2.  Tabulated  Responses  to  Question  Five 


5.  How  satisfied  are  you 

wi  th 

the 

number  of 

appoint- 

ment  scheduling  people 
in  your  area? 

N/A 

V/D 

D 

N/S/D 

S 

V/S 

0 

1 

2 

3 

4 

5 

Executive  Management 

44% 

33% 

0% 

1 1% 

1 1% 

0% 

Department  Chair 

0% 

29% 

71% 

0% 

0% 

0% 

Subtotal 

25% 

31% 

31% 

6% 

6% 

0% 

Supervisor 

0% 

14% 

14% 

14% 

43% 

14% 

Appointment  Clerk 

0% 

18% 

9% 

0% 

45% 

27% 

Subtotal 

0% 

17% 

1 1% 

6% 

44% 

22% 

COMBINED  TOTAL 

12% 

24% 

21% 

6% 

26% 

12% 

The  three  remaining  questions  were  not  addressed  as  they  are 
more  applicable  to  further  study  on  the  appointment  system 
other  than  accessibility  problems. 

Justification  for  further  analysis  was  identified  in  a 
talking  paper,  prepared  by  Major  Shields  in  July  1989  (24). 
It  was  a  compilation  of  standards  for  USAF  Medical  Center 
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appointment  systems.  The  standards  listed  were  derived  from 


both  local  and  DOD  recommendations. 

Project  goals .  As  a  study  progresses  it  is  sometimes 
easy  to  lose  sight  of  the  original  needs.  Ostrofsky 
suggests  preparing  a  statement  of  goals  to  provide  the 
designer  with  a  measurement  of  success  (14:33).  The 
following  goals  were  obtained  from  information  contained  in 
the  needs  analysis  and  the  talking  paper  (24) . 

1.  Time  on  hold  should  be  limited  to  less  than  5 
minutes . 

2.  Avoid  requiring  appointments  be  made  at  either 
the  start  or  end  of  each  month.  This  can 
result  in  a  smoother  workload  distribution. 

3.  Less  than  six  percent  of  all  calls  should 
encounter  a  busy  signal. 

4.  Downtime  should  be  limited  to  only  three  per¬ 
cent  of  total  monthly  hours  available. 

5.  At  least  70  percent  of  appointments  should  be 
made  before  the  patient  arrives,  thereby 
smoothing  workload  distributions. 

6.  Walk-in  rates  should  be  limited  to  10  percent 
of  patient  visits,  again  smoothing  workloads. 

7.  Average  appointment  transactions  should  last 
less  than  2.5  minutes. 

6.  Calls  for  information  should  not  exceed  10  per¬ 
cent  of  all  calls  received. 

9.  One  appointment  clerk  should  be  available  for 
every  2,000  monthly  calls. 

10.  An  average  of  1.5  telephone  lines  should  be 
available  for  every  appointment  clerk. 


20 


Identi f icatlon  and  Formulation  of  the  Problem 


The  purpose  of  this  stage  in  the  methodology  *is  to 
bound  the  needs  or  requirements  subjectively"  (14:35).  In 
doing  so  a  realistic  picture  or  concept  of  the  problem  is 
defined  that  allows  a  better  grasp  of  the  problem.  To  do 
this,  the  product ion/eonsumpt ion  cycle  is  examined  step  by 
step  in  an  attempt  to  identify  as  many  characterizing 
attributes  as  possible.  For  each  phase  in  the  life  cycle, 
consideration  is  given  to  what  is  input  and  what  results  as 
output . 

Desired  outputs  from  each  phase  may  not  always  be 
achievable  but  in  effect  serve  as  goals  to  strive  for. 
Undesired  output,  on  the  other  hand,  enumerates  the  unwanted 
aspects  of  the  phase  which  hopefully  can  be  eliminated  or  at 
the  very  least  minimized. 

The  input  side  of  a  phase  is  divided  into  what  is 
intended  and  what  is  subject  to  the  environment.  Intended 
inputs  include  items  or  processes  necessary  to  start  the 
particular  process.  Environmental  inputs,  describe  what 
already  exists  in  our  system  that  will  effect  this  phase  in 
the  life  cycle. 

To  define  these  boundaries  one  must  be  aware  of  how  an 
organization’s  goals  and  available  resources  may  affect  the 
project.  Essentially,  they  establish  the  outer  framework 
within  which  a  product  or  service’s  life  cycle  occurs. 
According  to  Ostrofsky,  a  product  encounters  four  phases 
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during  its  life-cycle:  production,  distribution, 
consumption/operation ,  and  retirement  (14:8).  These  phases 
are  described  in  general  and  also  how  they  relate  to 
appointment  systems. 

Production  Phase .  Initially,  the  transformation  of  raw 
materials  into  finished  goods  is  accomplished  during  the 
Production  phase.  Other  processes  may  occur  during  this 
phase  which  can  contribute  to  a  product's  future  success. 

For  this  study,  the  product  was  defined  as  an  available 
medical  appointment  from  any  of  the  clinics  utilizing  the 
centralized  appointment  system.  The  product  is  transformed 
from  the  raw  materials  or  Intended  inputs  into  an  available 
patient  appointment.  Table  3  is  an  outline  analysis  of  the 
Production  phase. 


Table  3.  Production  Activity  Analys i s 


INPUTS 

Intended 

Environmental 

1  . 

Provider's  schedule 

1 . 

Type  of  appt  system 

2  . 

Computer  online 

2. 

Method  of  appointment 

3. 

Appt  books  available 

booking 

OUTPUTS 

Desired 

Undes Ired 

1  . 

All  providers  available 

1 . 

Limited  providers 

2. 

Maximum  number  of  slots 

2. 

Incomplete  or  missing 

available 

medical  records 

3. 

No  prior  conflicting 

3. 

Limited  number  of 

comroi tments 

technicians  for  asst 

4  . 

Four  week  block  of  appts 

4  . 

Reduced  availability 

for  scheduling 

of  appointments 

22 


While  the  general  output  Is  an  appointment,  Its’  success  In 
the  Distribution  phase  Is  contingent  upon  the  output 
features . 

Distribution  Phase .  According  to  Ostrofsky,  this  phase 
'accomplishes  the  phase-ln  of  the  product  or  process  to  the 
ultimate  operators  or  consumers*  (14:11).  Before  a  product 
reaches  the  Intended  recipients  It  may  undergo  a  variety  of 
distribution  activities. 

The  distribution  phase,  outlined  In  Table  4,  Involves 
the  matching  of  eligible  patients  with  the  most  convenient 
appointment  slots. 


Table  4.  Distribution  Activity  Analysis 


INPUTS 

Intended 

Environmental 

1  . 

Equal  dally  and  weekly 

1.  Scheduling  system 

distribution  of  calls 

2.  Screen  or  appoint- 

2. 

Available  patient  info 

ment  book  layout 

3  . 

High  MTBF  for  hardware 

3.  Number  of  telephone 

and  software 

lines  available 

4  . 

Low  MTTR  for  hardware 

4.  MTBF  for  telephone 

5. 

Undistracted  clerks 

system 

6. 

Pleasant  work  environment 

5.  Length  of  time  slots 

7  . 

No  calls  for  Information 

e. 

No  busy  signals 

9. 

An  available  appointment 

OUTPUTS 

Desired 

Undesired 

1  . 

Minimizing  the  length 

1 

.  Unable  to  find  a  suit- 

of  time  on  hold 

able  appointment  time 

2. 

Favorable  appt  slot 

2 

.  Leaving  someone  on  hold 

located  for  patient 

longer  than  5  minutes 

3. 

Courteous  clerks 

3 

.  Difficulty  in  verifying 

4  . 

Availability  to  answer 

patient  eligibility 

phones 

4 

.  Missing  medical  records 
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This  can  involve  matching  patients  with  specifically 
requested  providers. 

Consumption/Qperation  Phase .  Once  a  consumer  has  the 
product  or  service,  it  is  then  consumed  or  operated  in  a 
manner  for  which  it  was  produced.  Consumer  satisfaction 
with  a  product  is  crucial  as  it  forms  the  basis  for 
continued  or  modified  production  of  the  product. 

The  Consumption/Operation  phase  identified  in  Table  5, 
entails  the  patient  who  arrives  for  their  scheduled  appoint¬ 
ment  and  meets  with  the  provider.  Alternatively  this  can 
also  Involve  the  patient  who  is  unable  to  meet  their 
appointment  time  and  must  either  cancel  or  reschedule. 


Table  5.  Consumptlon/Qperati on  Actl vi tv  Analysis 


INPUTS 

Intended 

Environmental 

1  . 

Provider  present  for 

1 . 

Inclement  Weather 

appointment  time 

2. 

Private  matters 

2. 

Provider  on  schedule 

3. 

Mobility  exercises 

3  . 

Patient  present 

4. 

Parking  available 

5. 

Location  of  clinic 

OUTPUTS 

Desired 

Undesired 

1  . 

Patient  arrives  prior 

1  . 

Patient  is  late 

to  scheduled  appointment 

2. 

Patient  fails  to  show 

2. 

Patient  uses  allotted 

for  appointment 

slot  of  time 

3. 

Patient  fails  to 

cancel  or  change  an 

appointment 

In  either  case  the  appointment  becomes  an  available  product 
and  can  be  recycled  back  to  the  distribution  stage  if 
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sufficient  time  exists,  otherwise  it  is  forwarded  to  the 


Retirement  stage. 

Retirement  Phase .  Often  overlooked  is  the  eventual 
need  for  product  retirement.  Unlike  services,  products  may 
become  technically  obsolete  in  a  few  years,  or  wear  out  and 
become  unrepairable.  Also,  thought  must  be  given  to  the 
eventual  demise  or  possible  future  service  that  can  be 
derived  from  a  product.  Such  extended  use  may  occur  through 
the  recycling  of  materials  or  retooling  of  worn  parts. 

The  final  phase  in  a  product’s  life,  is  the  planned 
activity  which  removes  it  from  the  life  cycle.  Table  6 
outlines  the  retirement  of  an  appointment,  which  usually 
occurs  after  the  patient  has  met  with  the  provider. 

Table  6.  Retirement  Activity  Analysis 


INPUT 


Intended 

1.  Send  back  to  distribution 
stage  if  possible 

2.  Substitute  walk-in  patient 
for  a  no-show 


OUTPUTS 


Desired  Undesired 

1.  Patient  is  seen  and  1.  Patent  reschedules 

treated  another  appointment 

2.  Appointment  is  left 

unfilled 


Environmental 
1.  Provider  is  made 
unavailable  by 
unexpected  mission 
requirements 


Synthesis  of  Solutions 


This  is  the  third  step  in  the  feasibility  study  and 
involves  "the  piecing  together  of  activities  to  integrate 
the  needs  established  and  defined  boundaries'  (14:54).  The 
sequence  of  events,  portrayed  in  the  product/consumption 
cycle  provides  a  flow  of  functions  which  help  meet  the  needs 
of  the  cycle,  as  illustrated  in  Figure  1.  As  mentioned 
earlier,  only  a  portion  of  the  entire  cycle  will  be 
considered  for  solution  as  illustrated  in  Figure  2.  That 
portion  was  identified  by  the  needs  analysis  as  a  more 
efficient  access  method  to  the  Medical  Center  appointment 
system. 

Although  an  inclusive  listing  of  concepts  may  include 
some  highly  questionable  approaches,  eliminating  any  concept 
before  it’s  been  evaluated  increases  the  risk  of  discarding 
the  optimal  candidate  system  itself.  As  a  result,  all  of 
the  following  concepts  meet  the  identified  need,  but  careful 
analysis  is  required  to  extract  the  best  one  overall. 

1.  Utilize  a  system  with  manual  appointment  booking 
and  standard  telephonic  access. 

2.  Utilize  a  system  with  automated  appointment  booking 
and  standard  telephonic  access. 

3.  Utilize  a  system  with  both  automated  appointment 
booking  and  computerized  telephonic  access. 

To  maximize  the  potential  solutions  to  the  problem  the 
concepts  must  be  'broken  down  into  their  corresponding 
subsystems  functions  and  activities'  (14:257).  Each 
subsystem  is  a  part  that  contributes  to  the  overall  function 
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PATIENT 


Figure  1.  Appointment  System  Activity  Life-Cycle 
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Figure  2.  Appointment  Booking  Process 
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of  the  system.  It  is  composed  of  many  activities  or 
alternatives,  each  of  which  contributes  to  the  completion  of 
an  appointment  call. 

Candidate  systems  are  created  by  combining  one 
subsystem  activity  or  alternative  from  each  subsystem 
function.  This  results  in  many  possibilities,  but  is  an 
attempt  to  capture  the  optimal  candidate  system. 

Three  concepts,  whl  h  provide  patients  with  access  to 
the  Medical  Center  appointment  system,  have  been  derived. 
Figure  3  illustrates  the  first  concept  which  relies  on  the 
traditional  manual  bookkeeping  method  of  making 
appointments.  This  requires  the  patient  to  either  call  for 
an  appointment  or  make  the  appointment  in  person.  The 
second  approach,  illustrated  in  Figure  4,  reflects  the 
current  Medical  Center  appointment  system.  Computerized 
appointment  books  are  accessed  to  enter  a  patient  into  the 
appointment  system.  In  Figure  5,  the  final  concept 
incorporates  computerized  telephone  automation  to  actually 
answer  the  phone  calls.  Such  a  device  could  complete  some 
of  the  calls  while  others  would  have  to  be  deferred  to  an 
actual  appointment  clerk. 

Although  all  three  concepts  are  potentially  capable  of 
satisfying  the  objectives,  Concept  I  relies  on  a  labor 
intensive  methodology.  Since  the  original  objective  was  to 
become  more  efficient,  such  a  concept  can  no  longer  be 
considered  viable  in  light  of  the  following  Information. 
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METHOD  OF 
CON  TACT 
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Figure  3.  CONCEPT  I:  Manual  Appointment  Booking 
with  Standard  Telephonic  Access 
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Figure  4.  CONCEPT  II:  Automated  Appointment  Booking 
with  Standard  Telephonic  Access 


METHOD  OF 
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Figure  5.  CONCEPT  III:  Automated  Appointment  Booking 
with  Computerized  Telephonic  Access 


During  fiscal  year  1986,  the  Medical  Center  saw  more  than 
363,000  clinical  outpatient  visits.  This  excludes  the  more 
than  28,000  inpatient  visits,  33,000  emergency  room  visits, 
and  26,000  Mental  Health  appointments  (22).  Such  a  work 
load  would  result  in  eleven  appointment  clerks  having  to 
manually  book  an  average  16.77  appointments  per  hour.  This 
average  fails  to  account  for  peak  workload  periods  and  does 
not  meet  the  goals  previously  stated.  Hence  Concept  I  was 
removed  from  any  further  consideration  and  attention  was 
focused  on  Concepts  II  and  III. 

A  breakdown  of  each  concept’s  subsystems  and 
accompanying  alternatives  are  listed  below.  Concept  II  is 
illustrated  in  Table  7.  Combinations  of  each  of  the 
alternatives  resulted  in  the  following  number  of  candidate 
systems ; 

ABCDEFQHI 

2x3x2x7x6x2x4x2x7=  56,448  candidate  systems 


Table  7,  Concept  I I  Candidate  Systems 


A.  Method  of 

Contact 

1 . 

Telephone 

2  . 

In  person 

B.  Hours  of 

Operation 

1  . 

One  8  hour  shift 

2. 

One  6  and  one 

4  hour  shift 

3. 

Two  6  hour  shifts 

C.  Response 

for  Deferred  Calls 

1  . 

Industry  standard  busy  signal 

2. 

Informational 

response 
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Table  7  continued. 


D.  Response  when  Put  on  Hold 

1 .  Music 

2.  Which  clinic  reached 

3.  Status  in  queue 

4.  Combine  1  and  2 

5.  Combine  2  and  3 

6.  Combine  1  and  3 

7.  Combine  1,  2  and  3 

E.  Length  of  Waiting  Queue 

1 .  0  callers 

2.  1  callers 

3.  2  callers 

4.  3  callers 

5.  4  callers 

6.  5  callers 

F.  Departmental  Organization 

1 .  Central ized 

2.  Decentralized 

G.  Location  of  Appointment  Office 

1 .  On  site 

a.  Single  Room 

b.  Multiple  Rooms 

2.  Off  Site 

a.  Single  Room 

b.  Multiple  Rooms 

H.  Clerk  Location 

1 .  Within  Cl inic 

2.  Outside  clinic 

I.  Method  of  Appointment  Confirmation 

1 .  Verbal  reply 

2.  Written  response 

a.  USPS  1st  class  letter 

b.  Base  distribution  (BITS) 

c.  FAX  transmission 

d.  Express  Mall 

e.  Electronic  Mail 

f.  Personally  delivered 
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Concept  III  is  outlined  in  Table  8.  Combinations  of  each  of 
the  alternatives  resulted  in  the  following  candidate 


systems : 

ABCDEFQH 

2x3x2x7x6x7x6x7=  146,176  candidate  systems 


Table  8.  Concept  III  Candidate  Systems 


A.  Method  of  Contact 

1 . 

Telephone 

2. 

In  person 

B.  Hours  of  Operation 

1  . 

One  8  hour  shift 

2  . 

One  8  and  one  4  hour  shift 

3  . 

Two  8  hour  shifts 

C.  Response  for  Deferred  Calls 

1  . 

Industry  standard  busy  signal 

2  . 

Informational  response 

D.  Response  when  Put  on  Hold 

1  . 

Music 

2  . 

Which  clinic  reached 

3. 

Status  in  queue 

4  . 

Combine  1  and  2 

5. 

Combine  2  and  3 

6  . 

Combine  1  and  3 

Combine  1,  2,  and  3 

E.  Length  of  Waiting  Queue 

1  . 

0  callers 

2. 

1  callers 

3. 

2  callers 

4  . 

3  callers 

5  . 

4  callers 

6. 

5  callers 

3S 


Table  8  continued. 


F.  Automation  Attendant 

1.  Prioritize  calls  (i.e.  Information, 
acute  care  need,  routine  appt) 

2.  Provide  information 

3.  Schedule  routine  appts 

4.  Combine  1  and  2 

5.  Combine  1  and  3 

6.  Combine  2  and  3 

7.  Combine  1,  2  and  3 

G.  Call  Disposition 

1.  Manual  Intervention  for  patient  assistance 

2.  Rotary  dial  phone  default 

3.  Acute  care  need 

4.  Combine  1  and  2 

5.  Combine  2  and  3 

6.  Combine  1  and  3 

7.  Combine  1,  2  and  3 

H.  Method  of  Appointment  Confirmation 

1 .  Verbal  reply 

2.  Written  response 

a.  USPS  1st  class  letter 

b.  Base  distribution  (BITS) 

c.  FAX  transmission 

d.  Express  Mail 

e.  Electronic  Mail 

f.  Personally  delivered 


Having  created  an  exhaustive  list  of  candidate 
systems,  the  focus  moved  to  eliminating  those  which  were 
clearly  infeasible  for  physical,  economic,  or  financial 
reasons . 


Screening  of  Candidate  Systems 

In  the  attempt  to  include  the  optimal  candidate  system 
no  candidates  should  be  eliminated  unless  proven  to  be 
physically  infeasible  to  operate  or  produce  as  depicted  in 
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Figure  6.  Consideration  should  also  be  given  towards  the 
constraints  and  bounds  established  during  the  input-output 
analysis . 


Physical  Real i sabi 1 i tv .  With  components  of  the 
subsystem  being  developed  separately  from  each  other  it 
becomes  necessary  to  eliminate  those  combinations  which  are 
not  physically  compatible.  The  combinations  listed  below  in 
Tables  d  and  10  for  both  concepts  were  removed  from 
consideration. 


Table  9.  Concept  I I  Physical  Incompatlbi 1 i ties 


Component 

Incompatible  With 

A2 

Cl  ,C2 ,Dl-6 ,El-6 

FI 

HI  ,H2 

F2 

aia,Glb,G2a,Q2b 

Net  Candidate  systems 

available:  56,448  -  90  =  56,358 

Table  10.  Concept 

III  Physical  Incompatibilities 

Component 

Incompatible  With 

A2 

Cl ,C2 ,Dl-6 ,El-6 ,Fl-6 ,G2-6 

Net  Candidate  systems  available:  148,176  -  3528  =  144,648 


Economic  Worthwhi leness .  Any  candidate  system  under 
review  must  be  able  to  provide  an  acceptable  level  of  return 
on  the  investment  made  on  it.  That  is,  'the  total  resources 
required  to  complete  the  design  of  the  candidate  must  be 


less  than  the  value  received  from  its  completion*  (14:58). 
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Figure  6.  Screening  of  Candidate  Systems  (14:66) 
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Unlike  manufactured  products,  productivity  from 
services  is  difficult  to  measure.  The  absence  of  a  profit 
motivated  unit  of  output  excludes  traditional  economic 
measurements  of  price-volume  relationship,  break  even 
analysis,  and  cost  estimates.  Although  appointment  quality 
was  mentioned  earlier  in  the  Desired  Output  portion  of  the 
Production  phase,  such  output  ’with  reference  to  quality  ... 
is  even  more  ephemeral  in  services  than  in  manufacturing' 

( 16:53)  . 

Most  activities  under  both  concepts  were  considered 
economically  worthwhile  to  develop.  The  only  exception  was 
the  activity  entitled  'Method  of  Appointment  Confirmation'. 
For  both  concepts  the  candidates  in  Tables  11  and  12  were 
excluded  from  further  consideration. 


Table  11.  Concept  1 1  Economic  Impractical 1  ties 


Component 

Incompatible  With 

I2c 

Al-2  ,Bl-3 ,Cl-2 ,Dl-7 , 

El-6, Fl-2,Gla-2b,Hl-2 

I2d 

Al-2 ,Bl-3 .Cl-2 ,Dl-7 , 

El-6, Fl-2,Gla-2b,Hl-2 

Net  candidate  systems  available: 

56,358  -  16,128  =  40,230 

Justification  for  the  exclusion  of  candidates  I2c  and 
H2c,  Facsimile  transmissions,  is  found  in  the  definition  of 
use.  A  FAX  is  designed  to  provide  a  timely  transmission  of 
Important  documents  when  nothing  else  is  available.  This 
precludes  its  use  from  anything  but  time-sensitive 
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Incompatible  Wi  th 


Table  12.  Concept  III  Econom 


Component  Incompatible  With 

H2c 

Al-2 ,Bl-3 ,Cl-2 ,Dl-7 
El-6 ,Fl-7 ,Gl-7 

H2d 

Al-2  ,Bl-3 ,Cl-2 ,Dl-7 
El-6 ,Fl-7 ,Gl-7 

Net  candidate  systems  available:  146,646  -  49,392  =  99,256 


documents.  Other  drawbacks  Include  the  need  for  both 
parties  to  either  own  or  have  access  to  a  FAX  and  any  long 
distance  charges  must  be  included  in  cost  estimates. 

Components  I2d  and  H2d ,  Express  Mall  service,  is  cost 
prohibitive  at  07.00  to  09.00  a  letter.  The  matter  of 
urgency  that  FAX  transmission  and  Express  Mail  solve  should 
not  exist  in  an  appointment  environment  whose  purpose  is  to 
plan  ahead. 

Financial  Feas ibi 1 i ty .  Even  though  a  candidate  system 
can  be  shown  to  be  an  optimal  solution,  a  lack  of  financial 
resources  can  eliminate  such  a  project  from  further 
development.  Consideration  should  be  given  to  the  nature  of 
the  systems  and  the  areas  of  funding. 

Tight  budget  constraints  have  made  many  DOD  projects 
extremely  limited  in  scope.  Even  so,  development  of  any  one 
of  the  candidate  systems  could  be  within  funding  require¬ 
ments  for  base  level  contract  administration.  Medical 
information  systems  local  purchase  levels  presently  justify 
purchases  less  than  015,000  (22).  Naturally,  implementation 
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would  be  contingent  upon  the  availability  of  funds  and 
willingness  to  develop  the  system. 

Conclusions  and  Summary 

The  need  for  more  efficient  access  to  the  appointment 
system  has  been  documented.  A  review  of  literature  has 
shown  the  need  for  and  significance  of  an  appointment 
system.  This  was  followed  by  a  discussion  on  the 
developmental  history  of  the  methodology  used  in  this 
research . 

Within  the  framework  of  the  organization,  input-output 
boundaries  were  established  to  define  the  limitations  and 
expectations  of  an  appointment  during  its  normal  life  cycle. 
This  study  then  focused  on  one  part  of  the  life  cycle,  that 
of  efficient  access  to  the  distribution  phase. 

Utilizing  standards  set  forth  by  the  USAF  and  the  DOD , 
established  goals  were  defined  to  achieve  efficient 
appointment  access.  To  solve  the  identified  need,  three 
solutions  or  concepts  were  developed.  Only  Concepts  II  and 
III  were  selected  for  further  analysis  to  maintain  the 
original  goal  of  efficiency  in  the  system. 

Activities  were  Identified  for  the  concepts  ana  broken 
out  into  subsystems,  from  which  candidate  systems  would  be 
derived.  With  several  thousand  candidate  systems  possible, 
an  initial  attempt  was  made  to  discard  candidate  systems 
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that  were  considered  physically,  economically,  or 
financially  infeasible  to  develop  further. 


Having  identified  the  problem  and  established  a  set  of 
possible  solutions,  the  next  concern  was  to  select  a  single 
'best'  candidate  system.  The  remaining  methodology,  set 
forth  in  Chapter  4,  establishes  one  system  that  "best*  meets 
the  needs  set  forth  in  Chapter  1. 
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IV.  Preliminary  Activities 


The  purpose  of  the  previous  chapter  was  to  establish 
the  concepts  that  would  meet  the  identified  needs.  This 
resulted  in  the  creation  of  many  candidate  systems.  With 
this  information,  the  goal  of  the  "Preliminary  Activities" 
section  is  to  find  the  "best"  solution  or  as  Ostrofsky 
wrote,  "identify  the  optimal  candidate  system  from  the  set 
of  candidates  already  defined"  (14:69). 

Preparation  for  Analysis 

Despite  efforts  to  eliminate  candidate  systems  for 
physical  or  economic  reasons  in  Chapter  III,  many  thousand 
remain.  The  more  candidate  systems  the  more  difficult  it 
becomes  to  subjectively  evaluate  them,  but  the  better  the 
chance  of  finding  the  "best*  solution. 

To  gain  a  new  perspective  on  the  candidates,  this 
activity  "provides  an  insight  into  the  set  of  candidates  and 
their  relationship  to  each  other  and  to  the  needs  analysis’ 
(14:74) .  Such  an  analysis  results  in  the  following 
obj  ecti ves . 

1.  An  increasing  awareness  of  the  nature  of  the 
criteria  to  be  met  by  the  emerging  system. 

2.  An  increasing  knowledge  of  the  nature  of  the 
candidate  systems  for  a  given  concept  and  the 
qualities  of  each  concept  in  the  broad  domain  of 
possible  concepts  available  to  meet  the  needs 
defined.  (14:74) 


43 


To  accomplish  the  above  objectives,  Ostrofsky  suggests 


grouping  candidate  systems  together  based  on  similar 
characteristics.  This  provides  renewed  consideration  of 
existing  criteria.  The  adequacy  of  each  of  the  groups  is 
identified  in  Tables  13  and  14,  by  preparing  a  list  of 
advantages  and  disadvantages  associated  with  each  grouping. 
The  areas  addressed  in  these  tables  are  established  to 
demonstrate  the  pros  and  cons  of  these  systems  and  how  they 
interact  with  the  optimization  process.  Further  discussion 
of  these  areas  follows  and  is  eventually  quantified. 

Def inition  of  Criteria 

To  evaluate  the  performance  of  a  candidate  system,  some 
scale  must  be  developed  to  measure  it  against.  A  set  of 
criteria  can  be  developed  to  quantitatively  choose  the  best 
candidate  system.  Criteria  can  be  measurable  quantities 
such  as  time  or  dollars,  or  exist  in  a  more  abstract  realm 
such  as  satisfaction  or  experience.  Ostrofsky  suggests 
extracting  criteria  from  information  in  the  needs  analysis 
and  listed  outputs,  both  desired  and  undesired,  from  the 
input/output  matrices.  The  clarity  and  precision  of  chosen 
criteria  are  essential  for  all  decision  elements  to  be 
adequately  considered.  Ostrofsky  presents  the  following 
criterion  development  theorem: 

Any  criterion  not  considered  will  not  be  included 

in  the  choice  of  the  optimal  candidate.  (14:80) 
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Table  13.  Concept  I I  Grouped  Candidates  by  Attributes 


Type  of 
Candidate 


Advantages  (26:392) 


Central ized 
Appointment 
System 


One  source  of  information  for  patients. 
Aval j.abi  1  i ty  of  dedicated  staff  to  answer 
telephones . 

Patients  can  make  multiple  clinic 
appointments  from  one  call. 

Compiles  workload  data  for  each  clinic 
served . 

Standardized  collection  of  patient 
appointment  information. 

More  precise,  coordinated  scheduling  of 
all  clinics  served. 


Disadvantages  (23:96) 

1.  Lack  of  flexibility  to  handle  acute  care 
and  emergency  situations. 

2.  Increased  communications  between  clinics 
and  appointment  area. 

3.  More  complex  system  to  provide  training  on, 

4.  Lack  of  familiarity  with  unique  clinic 
needs  to  properly  screen  patients. 

Advantages  (23:98) 


Decentra- 

1  . 

Greater  flexibility  in  scheduling  with. 

1  ized 

2. 

Greater  control  by  supervisors  over  clinic 

Appointment 

operations . 

System 

3  . 

Easier  to  make  follow-up  and  same  day 
appointments . 

4  . 

Reduce  training  time  for  appointment 
clerks . 

5. 

May  appear  to  patient  to  be  more 
responsive  to  needs. 

6  . 

Increased  ability  to  prioritize  patient 
appointment  needs. 

Disadvantages  (23:100) 


Lack  of  adequate  coverage  when  appointment 
clerks  are  absent. 

Separate  telephone  numbers  for  each  clinic 
make  it  more  difficult  for  patient  access. 
Difficulties  in  referring  patients  to 
other  clinics. 

Fluctuating  appointment  clerk  productivity. 
Increased  hardware  needs. 
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Table  14. 

Concept  III  Grouped  Candidates  by  Attributes 

Type  of 

Candidate 

Advantages 

Telephone 

1.  Saves  time  for  patient,  convenience. 

2.  Reduces  traffic  and  parking  problems. 

Disadvanta«{es 

1.  Connection  can  be  difficult  to  understand. 

2.  Long  distance  calls  for  some  patients. 

3.  Telephone  numbers  can  be  lost  or  forgotten. 

Advantaf^es 

In-person 

1.  More  personal  attention. 

2.  Written  confirmation  acquired  when  done. 

DisadvantafSes 

1.  May  require  larger  floor  space. 

2.  Longer  to  book  an  appointment  (23:100). 

3.  Long  lines  irritate  or  disperse  patients. 

Advantages 

Automated 

Telephone 

System 

1.  Minimal  maintenance  and  upkeep  costs. 

2.  Faster  and  more  efficient. 

3.  Perform  several  different  functions. 

4.  Continuous  operation  possible. 

Disadvantages 

1.  Breakdowns  may  shutdown  entire  system. 

2.  Automated  equipment  may  be  offensive  or 
frustrating  to  some  patients. 

3.  Programming  errors  may  result  in 
scheduling  nightmares. 

Advantages 

Manual 

Telephone 

System 

1.  Better  able  to  handle  emergencies. 

2.  Responsive  and  sympathetic  to  patient. 

3.  Better  able  to  respond  to  questions. 

Disadvantages 

1.  Absenteeism  and  scheduling  of  leave. 

2.  Wage  raises  (union  contracts). 

3.  Morale  problems. 
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Thus  it  becomes  crucial  to  painstakingly  identify  those 
elements  which  are  criterion,  otherwise  they  can  not  be 
considered  in  the  optimal  system  choice. 

The  following  criteria  were  chosen  after  evaluating 
their  ability  to  effectively  meet  the  original  requirements 
indicated  in  the  Feasibility  Study. 

Response  Ti me .  Response  time  refers  to  the  amount  of 
time  a  call  must  wait  before  being  answered.  The  length  of 
time  is  dependant  on  the  number  of  calls  already  in  the 
waiting  queue  and  the  duration  of  these  calls.  It  is 
affected  indirectly  by  the  time  of  day  and  week  a  call  is 
made.  In  an  interview  with  Major  Donald  Shields,  he 
indicated  response  time  to  be  a  very  important  facet  in 
achieving  patient  satisfaction.  Minimizing  this  waiting 
time  is  necessary  to  achieve  a  more  optimal  solution. 

Prioritizing .  Prioritizing  is  a  systematic  approach  to 
ranking  calls  based  on  urgency  of  need.  Prioritizing  calls 
is  a  very  subjective  practice  and  may  be  frustrating  to  a 
caller  who  is  incorrectly  placed  in  the  wrong  queue.  Any 
system  developed  would  rely  on  either  the  caller  or  the 
receiver  prioritizing  the  call  based  on  a  set  of  decision 
rules.  A  successfully  prioritized  call  will  be  delivered  to 
the  proper  queue  to  be  answered  in  turn. 

Costs .  Costs  refers  to  the  monetary  expenditures 
required  to  compensate  employees,  and  vendors  of  hardware 
and  software.  Budgetary  constraints  require  a  careful 
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evaluation  of  present  and  future  costs  when  reviewing  the 
performance  of  any  new  or  current  system. 

User  Fr i endl iness .  User  friendliness  is  the  degree  to 
which  a  user,  such  as  the  appointment  clerk,  feels  comfor¬ 
table  using  a  system.  The  more  at  ease  a  clerk  is  with  a 
system,  the  fewer  errors  should  be  committed  and  the  more 
responsive  they  are  to  customer  needs.  User  friendliness  is 
dependant  upon  the  complexity  of  the  system,  the  availabil¬ 
ity  of  helpful  features,  and  the  amount  of  training 
received . 

System  Availability.  System  availability  is  the 
operable  presence  of  all  needed  system  components  when  they 
are  requested  by  the  user  (25:461).  Possible  component 
failures  or  system  crashes  must  be  planned  for  by  ensuring 
adequate  backup  procedures  are  available.  Consideration 
must  be  given  to  adequate  repair  and  replacement  times. 

Criteria  Relative  Importance 

Having  more  than  one  criterion,  it  becomes  Important  to 
consider  the  relative  weight  for  each.  The  criteria 
described  may  not  be  of  equal  importance.  To  distinguish 
the  differences  requires  weighted  values  be  assigned  to  each 
cr 1  ter ia . 

These  rating  values,  summarized  in  Table  15  and 
referred  henceforth  as  a’(i),  may  be  derived  from  any 
information  acquired,  or  according  to  Ostrofsky,  ’even  an 
intuitive  evaluation  of  the  a’(i)  will  usually  yield  a 
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choice  closer  to  the  optimum  than  assuming  equal  Importance 
among  criteria"  (14:84).  To  weight  the  criteria, 
individuals  Involved  with  the  appointment  system  at  the 
Medical  Center  and  individuals  familiar  with  information 
systems  were  solicited  for  their  candid  ranking  of  the  five 
criteria.  The  following  heuristic  was  used  to  obtain  a 
weighted  set  of  a(i)  values  for  each  criterion. 

1.  Rate  each  criterion  on  a  scale  from  0  to  10 
(with  10  indicating  the  most  importance) . 

2.  Normalize  each  rating  by  the  total  of  all 
values . 

3.  Summation  of  a(i)  for  all  values  equals  1.0. 

4.  All  a(i)  values  must  range  from  0  to  1.0, 
inclusive.  (14:83) 


Table  15.  Criteria  and  Relative  Weights 


Criterion.  x(i)  Rating 

.  a’  (i) 

Weight  a(i) 

xl:  Response  Time 

9 

al:  0.24 

x2:  Prioritizing 

7 

a2:  0.19 

x3 :  Cost 

6 

a3:  0.16 

x4 :  User  Friendliness 

8 

a4:  0.22 

x5:  System  Availability 

7 

a5:  0.19 

1 .00 

Definition  of  Parameters 

Having  defined  the  criteria,  it  became  necessary  to 
establish  measurable  elements,  or  parameters  for  each 
criterion.  Collectively,  these  elements  assist  in  providing 
"the  major,  direct  source  of  the  candidate  systems 
quantitative  evaluation  by  the  criterion’  (14:86) .  Although 
some  criteria  can  be  directly  measured,  the  more  abstract 
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ones  require  the  identification  of  submodels  with  elements 
that  can  be  measured  and  used  to  evaluate  the  candidate 
system.  Design  or  planning  parameters  are  not  easily 
quantifiable.  The  relationships  between  criteria  must  be 
understood  in  order  to  obtain  the  most  realistic  and 
measurable  criterion  elements.  Ostrofsky  points  out  that  in 
the  development  process  some  elements  may  not  be  measurable 
from  'current  resources  but  which  are  considered  as 
contributors  to  the  meaning  of  the  criterion'  (14:88).  When 
this  occurs,  consideration  must  be  given  to  the  importance 
of  the  element.  If  crucial,  those  elements  must  be 
considered  by  some  manner  so  as  to  adequately  Judge  the 
candidate  system  performance.  Figure  7  provides  an 
illustration  of  the  relationship  between  parameters, 
submodels,  and  other  criterion  elements. 

In  Table  16,  each  criterion  and  associated  element  or 
parameter  is  listed.  Assigned  to  each  element  is  an 
alphabetic  letter  code  which  describes  a  specific  design 
parameter  outlined  in  Table  17.  Collectively  these  codes 
were  used  to  group  and  quantify  each  criterion. 

Having  completely  listed  the  criterion  elements, 
Ostrofsky  encourages  the  planner  to  Inspect  them  for 
'consistency  and  completeness*  (14:90).  In  doing  so, 
similar  meanings  and  redundancies  can  be  eliminated 
resulting  in  a  minimal  number  of  elements  necessary  to 
describe  the  criteria.  It  is  equally  Important  to  be  as 


Figure  7.  Elements  of  a  Criterion  For  a 

Set  of  Candidate  Systems  (14:88) 


Table  16.  Criteria .  Elements .  and  Parametric  Codes 


Criterion 

Elements 

Code 

xl  . 

Response  Time 

Number  of  calls  on  hold 

a 

1 

Transaction  length  per  call 

a 

x2  , 

Prioritizing 

Number  of  Incoming  Calls 

a 

Conciseness  of  decision  rules 

d 

Factors  of  fatigue  on  individual 

d 

prioritizing  calls 

Number  of  incorrectly  triaged 

a 

calls 

x3  , 

Costs 

Clerk  wages 

a 

Individual  equipment  costs 

a 

Primary  hardware  costs 

a 

Software  costs 

a 

x3  , 

Costs 

Maintenance  contract  fees 

a 

Support  personnel  wages 

a 

Support  hardware  and  supplies 

a 

x4  , 

User 

Presence  of  user  requested  help 

a 

Fr iendl iness 

features 

Available  quick  reference  guides 

a 

Ability  to  start  over  without 

a 

having  to  backout 

Number  of  training  hours  received 

a 

Complexity  of  software 

d 

x5  , 

System 

MTBF  (Mean  Time  Between  Failure) 

a 

Aval lability 

MTTR  (Mean  Time  To  Repair) 

a 

Table  17.  Code  Def Inltlons 


Code  Type  of  Element 

a  Directly  measured. 

b  Measured  from  a  model  that  Includes 

some  of  the  a’s. 

c  Completely  included  In  other 

elements . 

d  Not  measurable  within  existing 

resources . 


complete  as  possible.  An  exhaustive  list  of  elements  is 
just  as  crucial  as  the  criteria  listing,  for  exclusions  of 
elements  may  not  be  included  in  later  decisions  concerning 
the  candidate  systems.  Ostrofsky  also  makes  mention  of  the 
need  for  element  compactness  which  combines,  when  possible, 
unit  values  of  elements  (14:91). 

Having  considered  the  aforementioned  suggestions,  the 
following  elements  were  defined. 

Yl:  Humber  of  calls  on  bold.  This  parameter  is 

limited  by  the  number  of  calls  that  can  be  stacked  in  the 
queue . 

Y2:  Transaction  length  per  call.  The  length  of  time 

it  takes  to  complete  an  appointment  request  is  measured  by 
this  parameter. 

Y3 ;  Humber  of  incoming  calls.  This  parameter  is  a 
measure  of  calls  that  arrive  at  the  hospital  and  are  either 
answered,  put  on  hold,  or  deflected  with  a  busy  signal. 

Y4:  Humber  of  incorrectly  triaged  calls.  This 

parameter  refers  to  the  number  of  calls  that  are  incorrectly 
identified  as  to  priority.  Subsequently  the  calls  are 
placed  in  the  wrong  category  for  response. 

Y5:  Clerk  wages.  This  is  the  appointment  clerks’ 

hourly  salary. 

Y6:  Individual  equipment  costs.  This  parameter 

includes  items  such  as  office  furniture,  standard  office 
supplies,  headsets,  and  phones. 
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Y7 :  Prlmapy  hardware  costs.  This  parameter 

encompasses  the  computer  and  related  hardware  that  are 
dedicated  to  the  appointment  system.  This  includes  CRT 
screens,  keyboards,  drive  units,  and  CPUs. 

Y8 :  Software  costs.  Whatever  computer  instruction  is 

required  to  satisfy  the  needs  of  the  appointment  system  is 
included  under  this  parameter.  This  cost  may  be  an  outright 
purchase,  site  license,  lease,  or  rental  fee. 

Y9:  Maintenance  contract  fees.  This  cost  covers  the 

contract  necessary  to  provide  maintenance  and  repair  of 
acquired  hardware  and  software. 

YIO:  Support  personnel  wages.  Unlike  the  wages  for 

clerks,  this  parameter  may  only  be  a  ratio  of  the  work 
actual  obtained  from  the  personnel.  Support  personnel  may 
be  involved  in  other  duties  apart  from  the  appointment 
system. 

Yll:  Support  hardware  and  supplies.  This  parameter 

accounts  for  the  miscellaneous  equipment  needed  for  support 
of  the  appointment  system.  This  would  include  printers, 
paper,  ribbons,  disks,  and  communication  lines. 

Y12:  Presence  of  user  requested  help  features.  This 

parameter  allows  appointment  clerks  to  customize  their 
appointment  system  display  with  any  needed  help  screen 
overlays  available. 

Y13:  Ability  to  start  over  without  having  to  backout. 

This  allows  a  clerk  to  quickly  escape  from  a  potential 
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error-making  situation  instead  of  having  to  backout  level  by 
level  from  the  current  screen. 

Y14:  Available  quick  reference  guides.  The  parameter 

refers  to  the  availability  of  instructional  aids  that  can  be 
quickly  referenced  while  operating  a  system.  Such  guides 
may  include  continuity  folders  that  outline  specific  tasks 
and  available  options,  keyboard  templates,  and  quick 
reference  sheets  available  at  a  glance.  Availability  of 
such  tools  may  be  in  the  form  of  integrated  software 
features  or  preprinted  reference  guides. 

Y15:  Number  of  training  hours  received.  This  parameter 
quantifies  the  number  of  training  hours  a  clerk  receives 
involving  the  specifics  of  the  appointment  system.  Training 
can  be  generalised  or  it  may  involve  specific  clinic  detail. 

Y16:  IfTBP  (Mean  Time  Between  Failure).  This  parameter 

is  ‘a  measure  of  the  average  amount  of  time  a  given 
component  may  be  expected  to  operate  before  failing' 

(25:539).  Such  a  measurement  reflects  the  reliability  and 
dependability  of  the  acquired  equipment.  Generally  MTBF 
values  are  given  by  the  manufacturer.  Selecting  equipment 
with  high  MTBF  values  should  extend  operation,  increase 
confidence  in  the  system,  and  increase  productivity  by 
having  fewer  down  times. 

Y17:  MTTR  (Mean  Tiske  to  Repair).  This  measure  is  'the 

average  time  required  to  fix  a  failed  component  ...  and 
restore  it  to  service'  (25:539,530).  The  duration  of  repair 
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time  can  vary  depending  on  the  number  of  affected  systems 
and  their  complexity.  MTTR  can  provide  a  baseline  estimate 
for  contingency  planning  for  the  length  of  time  manual  back 
up  services  may  be  required.  Generally,  the  MTTR  figure  is 
available  from  the  manufacturer. 

Some  elements  were  coded  "d*  as  limited  resources 
constrain  the  development  of  parameters  to  measure  these 
elements.  Elements  coded  "d"  that  describe  the  second 
criterion  (Prioritizing)  include;  conciseness  of  the 
prioritizing  decision  rules  and  the  factors  of  fatigue  on 
the  individual  who  is  prioritizing  the  calls.  Complexity  of 
software  was  also  coded  "d*  and  relates  to  Criterion  x4 , 
(User  Friendliness).  This  element  considers  the  nature  of 
the,  sof tware ,  realizing  the  more  complex  the  software 
operation  the  less  likely  it  is  to  be  truly  user  friendly. 

Criterion  Mode  ling 

This  section  begins  to  quantify  previously  subjective 
values.  Having  established  a  set  of  criteria  and  parame¬ 
ters,  it  now  becomes  possible  to  identify  one  candidate 
system  'whose  performance  as  identified  on  the  scale 
emerging  from  the  criteria  is  better  than  comparable 
performance  from  any  of  the  remaining  candidate  systems' 
(14:95).  Not  only  does  a  'best'  candidate  system  emerge,  it 
is  ranked  along  with  other  systems  being  nsidered. 
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While  the  criteria  identified  are  not  directly  measur¬ 
able,  the  parameters  selected  are  combined  to  describe  each 
cr i ter i a . 

This  effort  is  limited  by  the  subjectivity  involved  in 
development  of  mathematical  functions  that  relate  criterion 
to. the  relative  parameters.  Additionally  estimates  must  be 
made  of  minimum  and  maximum  values  for  each  parameter  as 
they  relate  to  their  criterion. 

Response  Time .  Quantitatively  described  as  the  amount 
of  time  transpired  until  the  call  is  answered.  This 
criterion  is  defined  by  the  following  elements; 

Yl:  Number  of  calls  on  hold 

Y2 :  Transaction  length  per  call 

The  resulting  function  is  illustrated  in  Appendix  C.  This 
function  is  of  an  increasing  linear  nature  and  relates  the 
product  of  the  two  parameters  as  the  total  amount  of  time 
existing  until  a  response  can  be  made.  The  resulting 
equation  for  the  criterion  becomes: 

xl  =  (Yl) (Y2) 

Prioritizing .  This  function  quantifies  the  success  of 
any  prioritizing  protocol  employed.  Success  is  dependant 
upon  all  the  elements  identified  in  Table  16  but  only  the 
following  were  measurable. 

Y3 :  Number  of  Incoming  calls 

Y4 :  Number  of  Incorrectly  triaged  calls 

As  the  number  of  incoming  calls  increases,  the  probability 
of  successfully  prioritizing  a  call  decreases.  Aggravating 
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this  problem  is  the  number  of  calls  incorrectly  prioritized. 
The  resulting  function  is  a  downward  sloping  curve  that 
approaches  zero.  This  model,  illustrated  in  Appendix  C,  is 
represented  by  the  following: 

x2  =  -0.5  (Y3  +  Y4)* 

Costs .  This  linear  function  is  a  summation  of  both 
direct  and  indirect  costs.  The  criterion  is  defined  by  the 
following  elements; 

Y5 :  Clerk  wages 

Y6 :  Individual  equipment  costs 
Y7 :  Primary  hardware  costs 
Y8:  Software  costs 
Y9:  Maintenance  contract  fees 
YIO:  Support  personnel  wages 
Yll;  Support  hardware  and  supplies 

Those  costs  considered  indirect  are  valued  at  proportionate 

rates  and  included  in  the  total  costs  required  to  maintain 

the  appointment  system  in  an  operable  mode.  The  function  is 

also  depicted  in  Appendix  C  and  the  resulting  equation  for 

the  criterion  function  becomes; 

x3  =  (Y5  +  Y6  +  Y7  +  Y8  +  Y9  +  YIO  +  Yll) 

User  Fr lend  1 iness .  This  function  is  particularly 

difficult  to  quantify  as  the  friendliness  of  a  system  to  a 

clerk  may  be  perceived  differently.  The  criterion  is 

defined  by  the  following  elements; 

Y12;  Presence  of  user  requested  help  features 
Y13:  Available  quick  reference  guides 

Y14:  Ability  to  start  over  without  having  to  backout 
Y15;  Number  of  training  hours  received 
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Optional  features  are  measured  as  either  being  present  or 

not.  The  sum  of  these  features  is  divided  by  the  number  of 

training  hours  required  by  either  the  clinic  or  manufacturer 

to  make  a  user  proficient  with  the  system.  Complex  systems 

may  have  all  the  features  of  a  simpler  system  but  will 

require  more  training  hours  to  become  proficient.  The 

result  is  a  decreasing  value  in  user  friendliness  for 

Increasing  hours  of  training.  The  function  is  shown  in 

Appendix  D  and  the  resulting  equation  is; 

x4  =  (Y12  +  yi3  +  Y14)/  Y15 

System  Aval lability.  To  the  user  or  clerk,  this 

function  describes  the  availability  of  a  system  at  any  given 

time.  The  criterion  is  defined  by  the  following  elements: 

Y16:  MTBF  (Mean  Time  Between  Failure) 

Y17:  MTTR  (Mean  Time  To  Repair) 

Using  MTTR,  MTBF,  and  time  interval.  Nickel  suggested  the 

following  probability  function  to  determine  system 

aval lability. 

- (a+b) t 

A(t)  =  [a/(a+b)]  +  [b/(a+b)]  e 
Where  a  =  1/(MTTR),  b  =  1/(MTBF),  e  is  the  natural 
logarithm,  and  t  is  a  time  interval  (12:80;  25:462) .  The 
result  of  this  function  is  the  number  of  times  a  system  will 
be  available  during  a  specified  time  period.  Stamper  notes 
that  as  the  time  interval  increases  'the  exponential  term 
approaches  zero  and  becomes  insignificant'  (25:462). 

The  resulting  function  is  illustrated  in  Appendix  D  and  the 


59 


resulting  criterion  equation  function  as  suggested  by 
Stamper  becomes: 

x5  =  (Y16)/(yi6  +  Y17) 

Most  systems,  however,  are  comprised  of  various  components, 
each  of  which  have  their  own  MTBF  and  MTTR  values.  Total 
system  availability  then  becomes  a  product  of  all  component 
availability  as  illustrated  below  (12:80). 

A(s)  =  A(l) A(2)  . . .  A(n) 

where 

A(s)  =  Total  system  availability 
A(l)  ...  A(n)  =  Individual  component  availability 

The  preceding  equations  and  corresponding  functions 

illustrate  a  pattern  of  behavior  associated  with  each  of  the 

criteria.  Having  developed  the  function,  the  next  step  was 

to  establish  an  acceptable  range  of  values  for  which  the 

individual  criteria  can  assume. 

Formulation  of  the  Criterion  Functions 

Earlier,  in  Chapter  1,  a  problem  identification 
analysis  was  performed  to  subjectively  bound  the  production/ 
consumption  cycle  inputs  and  outputs.  This  process  of 
bounding  continues  after  defining  the  criterion  functions, 
to  limit  the  parameter  ranges. 

After  consulting  with  Major  Shields  (22),  reviewing  his 
talking  paper  on  standards  (24),  and  Colonel  Stuart's 
findings  (26:394),  the  following  parameter  ranges  were 
established  based  upon  current  trends,  existing  data,  and 


preferred  outcomes.  Formulation  of  parametric  ranges,  is 
extremely  important  considering  that  the  wider  the  range  the 
more  candidates  are  included  and  the  narrower  the  fewer. 
While  debating  the  minimum  and  maximum  values  for  each 
parameter,  it  was  understood  that  candidate  systems  yielding 
values  outside  of  this  boundary  are  not  considered. 

The  first  range  considered  involves  the  criterion 
elements  and  is  laid  out  in  Table  18.  The  17  parameters  are 
listed  with  their  respective  minimum  and  maximum  values. 

Having  defined  an  acceptable  range  of  values  for  each 
of  the  parameters,  it  was  possible  to  construct  a  similar 
table  (Table  19)  of  minimum  and  maximum  values  for  each 
criterion.  Using  the  criterion  functions  developed  earlier, 
the  minimum  parameter  values  are  placed  into  the  equations, 
with  the  result  of  a  single  criterion  minimum. 

Maximum  criterion  values  are  arrived  at  in  the  same  manner. 

A  realistic  set  of  values  had  to  be  determined  and  the 
tables  do  provide  a  realistic  range  of  values  outside  of 
which  no  candidate  system  was  considered. 

Combining  Criteria  into  one  Function .  Having 
completed  Table  18,  it  was  possible  to  synthesize  a  function 
which  Included  all  the  criteria.  The  result  is  a  single 
equation  that  provides  a  scaled  performance  value  for  each 
candidate  system. 

Having  developed  the  criteria  Independently  of  one 
another,  it  became  apparent  that  some  method  of  handling  the 
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Parameter  Y(k) 

Y(k)  Min 

Y(k)  Max 

Mean 

Y1  : 

Number  of  calls  on 
hold 

0  calls 

3 

1 . 5 

Y2: 

Transaction  length 
per  call 

1  min 

5 

3.0 

Y3: 

Number  of  Incoming 

Calls  per  hour 

0  calls 

504 

252 

Y4  : 

Number  of  incorrectly 
triaged  calls 

0  calls 

10 

5.0 

Y5  : 

Clerk  wages 

S 12 , 195 

*13,248 

*12,721 

Y6  : 

Individual  equipment 
costs/clerk 

*500 

*5,000 

*2,750 

Y7: 

Primary  hardware  costs 

*5,000 

*500K 

*252,500 

Y8; 

Software  costs 

*5,000 

*100K 

*52,500 

Y9: 

Maintenance  contract 
f  ees /year 

*150K 

*400K 

*225K 

YIO  : 

Support  personnel  wages 

*2,000 

*10K 

*6,000 

Y1 1  : 

Support  hardware  and 
suppl ies/year 

*500 

*3,400 

*1 ,950 

Y12  : 

Presence  of  user 
requested  help  features 

0 

1 

0.5 

Y13: 

Available  quick  refer¬ 
ence  guides 

0 

1 

0.5 

yi4  : 

Ability  to  start  over 
w/o  having  to  backout 

0 

1 

0.5 

Y15  : 

Number  of  training  hours 
received 

5  days 

20 

12 . 5 

Y16: 

MTBF  (Mean  Time  Between 
Failure) 

480  hrs 

16,640 

8,560 

Y17: 

MTTR  (Mean  Time  To 
Repair) 

1  hour 

6  hours 

3.5 

Table  19.  Range  of  Criteria  x ( i ) 


Criterion  x(i) 

Min 

Max 

Mean 

xl  : 

Response  Time 

0 

15 

7.5 

x2: 

Prioritizing 

-50 

-127,008 

-63,529 

x3: 

Cost 

175 , 195 

1,031 ,648 

603,422 

x4  : 

User  Friendliness 

0.0 

0.60 

0.30 

x5 : 

System  Availability 

0.9979 

0.9996 

0.9988 

units  of  each  criterion  must  be  implemented.  Ostrofsky 
warns  that , 
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unless  some  method  for  relating  the  sensitivity  of 
the  unit  value  of  x(l)  with  the  unit  value  of  x(2) 
is  used,  the  resulting  combination  of  these 
criteria  will  not  be  meaningful.  (14:113) 

Since  the  criterion  function  is  a  method  of  comparing  the 

candidate  systems  against  each  other,  it  not  only  seems 

fitting  but  essential  that  their  units  be  consistent  with 

each  other.  Ostrofsky  suggests  'identifying  criterion 

performance  as  a  fraction  of  the  allowable  range  for  that 

criterion'  (14:114).  This  unitless  fraction,  must  be 

multiplied  by  the  respective  criterion’s  relative  weight 

(previously  identified  in  Table  15).  This  criterion 

function  is  Equation  1,  which  will  yield  a  criterion 

performance  value  based  on  the  parametric  values  submitted. 


where 


n 

CF(o«) 
i  =  l 


2  a(i) 


X ( i )  -  X ( i ) min 


X ( i ) max  -  x ( i )  min 


(1) 


0.0  5  CF(o<)  i  1.0 


CF(a) 

n 

a(l) 

X  (i) 

X ( i ) min 
X ( 1 ) max 


Criterion  value  for  each  a  (candidate  system) 

Number  of  individual  criteria 

Relative  weight  assigned  to  ith  criterion 

ith  criterion  value 

Minimum  value  for  1th  criterion 

Maximum  value  for  ith  criterion 


Substituting  the  appropriate  values  into  Equation  1,  results 


in  the  following  criterion  function. 


CF 


. 24  »  [ (Y1«Y2) /15] 

+  .19  »  [ (-0. 5» (Y3  +  Y4)* +50) / (-127058)  ] 

+  .16  «  { [ (Y5+Y6+Y7+Y8+Y9+Y10+Y1 1) -175195]/856453} 

+  .22  »  {[ (Y12+Y13+Y14)/Y153/.60) 

+  .19  »  {[ (Y16/(Y16+Y17))-. 9979]/. 0017) 
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Analysis  of  the  Parameter  Space 

Although  a  complex  criterion  function  was  developed,  a 
understanding  of  all  parameter  interactions  remained 
elusive.  Attaining  complete  knowledge  of  a  system  may  not 
be  practical ,  but  understanding  as  much  of  the  design  space 
as  possible  is.  Without  a  complete  understanding  of 
criteria  interactions,  certain  risks  and  uncertainties  are 
present  and  should  be  minimized.  Although  the  choice  of 
criteria  has  been  based  on  a  variety  of  assumptions, 
no  analysis  has  yet  assessed  their  degree  of  interaction. 
Ostrofsky  summarizes  by  stating  that  "the  accuracy  of  the 
entire  activity  is  dependent  on  many  facets  of  the  s.ituation 
which  may  not  be  included  in  the  quantification  process' 
(14:119). 

After  examining  the  criteria  it  was  felt  that  no 
substantive  interactions  were  apparent  between  criteria. 
Analysis  in  the  following  areas  will  help  to  understand  and 
minimize  uncertainties  and  risks  in  the  design-planning 
space . 

Sens! tivity  Analysis .  The  purpose  of  this  analysis  is 
to  identify  the  rate  of  change  or  sensitivity  of  parameters 
to  the  criterion  function.  Ostrofsky  points  out  that  such 
an  analysis  can  point  out  'limitations  of  a  given  concept  or 
(candidate  system)'  (14:121).  The  result  is  additional 
Insight  in  to  the  planned  system  and  eventually  a  better 
design.  Those  parameters  altered,  will  result  in  varying 
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degrees  of  change  on  the  criterion  function  value.  In 
short , 

this  will  facilitate  the  decision  making  concerning 
any  changes  or  adjustments  that,  for  various  reasons, 
may  need  to  be  made  to  the  optimal  candidate  system 
when  identified.  (14:288) 

To  conduct  this  analysis,  two  computer  runs  were  initiated. 
The  first  output,  generated  by  the  computer  code  in  Appendix 
E,  was  recorded  in  Table  20  and  demonstrated  a  change  when 
one  parameter  was  varied  by  a  small,  fixed  amount  (five 
percent)  while  the  remaining  parameters  were  held  constant. 
As  an  initial  investigative  value,  the  mean  of  each 
parameter  range  was  chosen  as  a  constant  from  which  to  work. 
Minimum  and  maximum  values  were  excluded  as  starting  points 
because  of  the  possibility  for  extreme  sensitivity 
reactions . 

After  computing  the  initial  criterion  function  value 
'CFV)  for  the  parameter  means,  one  parameter  was  increased 
by  five  percent.  Next  the  CFV  was  recalculated  with  the  new 
parameter  value,  the  difference  in  CFVs  noted,  and  a 
corresponding  rate  of  change  was  observed. 

The  results  showed  parameters  Y1  (number  of  calls  on 
hold)  and  Y2  (transaction  length  per  call)  with  the  highest 
percentage  change  in  the  CFV.  This  analysis  Indicates  that 
parameters  Y1  and  Y2  are  the  most  sensitive  to  change  in 
relation  to  the  CFV.  This  indicates  that  any  variation  or 
change  of  these  parameters  should  be  considered  carefully  as 
they  will  produce  the  greatest  effect  on  the  CFV. 
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Table  20.  Sensitivity  Analysis 


Y(k) 

CFg 

Yk  mean 

Change 

in  IK 

CF'a 

Changes 
in  CFa  Pet 

Y1 

0.3798 

1.5 

1 . 58 

.3834 

.0033 

.94 

Y2 

0 . 3798 

3.0 

3. 15 

.3834 

.0033 

.94 

Y3 

0.3798 

15.0 

15.75 

.3802 

.0004 

.  11 

Y4 

0.3798 

5.0 

5.25 

.3802 

.  0004 

.  1 1 

Y5 

0.3798 

12721.0 

13357.05 

.3799 

.0001 

.03 

Y6 

0.3798 

2750.0 

2887.50 

.3798 

.  0000 

.00 

Y7 

0.3798 

252500.0 

265125.00 

.3821 

.0023 

.62 

Y8 

0.3798 

52500.0 

55125.00 

,3803 

.0004 

.  12 

Y9 

0.3798 

225000.0 

236250.00 

.  3819 

.0021 

.  55 

YIO 

0 . 3798 

6000.0 

6300.00 

.3798 

.0000 

.01 

Y1 1 

0 . 3798 

1950 . 0 

2047.50 

.3798 

.  0000 

.00 

Y12 

0 . 3798 

0.5 

0.53 

.3805 

.0007 

.  19 

Y13 

0.3798 

0.5 

0.53 

.  3805 

.0007 

.  19 

Y14 

0 . 3798 

0.5 

0.53 

.3805 

.0007 

.  19 

Y15 

0 . 3798 

12.5 

13. 13 

.  3777 

-  .  0021 

-  .  56 

Y16 

0 . 3798 

8560.0 

8988 . 00 

.  3819 

.0021 

.  57 

Y17 

0 . 3798 

3.5 

3 . 68 

.  3775 

-  .  0023 

-  .61 

Parameters  Y7 ,  Y9 ,  Y15,  Y16,  and  Y17  also  had  large 
percentage  differentiations  and  therefore  appropriate  care 
is  required  when  making  adjustments  to  these  parameters  as 
well. 

Rather  than  using  a  single  variance  in  the  parameter, 
the  second  analysis  varied  each  parameter  over  a  range  of 
values  using  a  fixed  increment  of  five  percent.  As  before 
all  other  parameters  were  held  constant  about  their  means. 
Changing  the  parameter's  amount  over  a  range  of  values 
provided  an  investigation  of  CFV  sensitivity  at  varying 
levels.  While  the  computer  run  (program  code  in  Appendix  F) 
produced  eleven  sensitivity  evaluations  for  each  parameter 
(Appendix  Q) ,  Table  21  summarizes  for  each  parameter  the 


least  and  greatest  percentage  change  in  the  CFV. 
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Table  21.  Sensitivity  Range  Summary 


Y(k) 

Least  Percentage 
Change  in  CFo 

Greatest  Percentage 
Change  in  CFa 

Y1 

0.94 

5 . 23 

Y2 

0 . 94 

5.23 

Y3 

0.11 

0.69 

Y4 

0.11 

0.69 

Y5 

0.03 

0. 17 

Y6 

0.00 

0.04 

Y7 

0.62 

3.43 

Y8 

0 . 12 

0.59 

Y9 

0.55 

3.05 

YIO 

0.01 

0.08 

Yll 

0.00 

0.03 

Y12 

0. 19 

1 . 06 

Y13 

0. 19 

1 . 06 

Y14 

0. 19 

1.06 

Y15 

0.56 

3.38 

Y16 

0.57 

3.52 

yi7 

0.60 

3.33 

As  expected,  parameters  Y1  and  Y2  again  showed  the  least  and 
greatest  percentage  changes,  reinforcing  their  significant 
effect  on  the  CFV. 

Compatibl 1 ity  Analysis .  Having  gained  useful 
information  from  the  sensitivity  analysis,  the  challenge 
still  remained  to  assemble  the  proposed  system  into  a 
functional  entity,  'Compatibility  is  the  orderly,  eificient 
integration  and  operation  of  elements  in  the  system' 

(14:291).  Therefore,  the  purpose  of  the  compatibility 
analysis  was  to  ensure  the  subsystems  would  work  together  in 
a  worthwhile  manner.  Ostrofsky  suggests  'examining  the 
design  parameters  and  noting  which  have  the  least  effect  on 
the  total'  CFV  (14:127).  The  effect  of  this  occurs  when 


67 


needed  changes  are  identified,  then  the  alteration  of 
parameters  will  commence  with  those  previously  Identified  as 
having  the  least  effect  on  the  CFV. 

Referring  to  Tables  20  and  21,  both  show  the  following 
parameters  to  be  the  least  sensitive: 

Y5 :  Clerk  wages 

Y6 :  Individual  equipment  costs/clerk 
YIO:  Support  personnel  wages 
Yll:  Support  hardware  and  supplies/year 

If  changes  to  the  parameters  are  required  to  ensure  system 

compatibility,  then  these  parameters  should  be  adjusted 

first  since  they  produce  the  least  effect  on  the  CFV. 

Should  these  parameters  not  produce  the  desired  effect  then 

consideration  must  be  given  to  other  parameters  with  greater 

sensitivity  values,  until  a  suitable  solution  can  be  found. 

Stability  Analysis .  Identifying  changes  and  making 

corrections  to  a  system  will  help  ensure  its  success  in  the 

field.  Understanding  what  the  system  will  actually  do  in 

its  enyironment  is  the  subject  of  a  stability  analysis. 

As  stated  earlier,  an  appointment  system  must  meet  the 

needs  of  both  patients  and  providers.  When  uneven  workloads 

cause  system  stress,  eventually  component  failure  may  occur. 

Routine  maintenance  may  be  able  to  prevent  such  breakdowns, 

but  adequate  system  planning  would  aid  the  designer  in 

adjusting  for  such  subsystem  Interactions  that  may  make  the 

proposed  system  susceptible  to  failure.  This  in  effect  is 

preventative  thinking  or  actions  that  will  preclude  the 

effects  of  unforeseen  disasters  or  untimely  events. 
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Ostrofsky  depicts  such  system  Interactions  in  an  'm+l' 
Euclidean  space,  where  m  is  the  number  of  parameters 
(14:129).  Thus  the  appointment  system  model  becomes  an  18 
dimensional  surface  with  the  18th  dimension  being  the  value 
of  the  criterion  function  itself.  For  each  interaction 
between  parameters  there  is  17  levels,  each  corresponding  to 
the  number  of  parameters  involved.  Such  a  depiction  is 
extremely  complex  and  beyond  the  scope  of  normal  analysis. 

In  a  more  understandable  3-dimen3ional  example,  stable 
interactions  may  be  represented  by  hills  or  maximum  CFVs  and 
less  stable  interactions  by  valleys  or  minimum  CFVs.  Other 
valid  CFVs  are  depicted  by  flatter  sheer  planes  in  between 
the  hills  and  valleys  where  interactions  are  highly 
unstable.  Ostrofsky  points  out  that  having  an  understanding 
for  the  shape  of  these  hills,  valleys,  and  planes  will  aid 
the  designer  in  his  awareness  of  'conditions  which  might 
cause  failure  or  major  malfunctions’  (14:129). 

Since  the  criterion  function  itself  was  constructed 
mathematically  Instead  of  by  practical  observation,  the 
function  itself  is  a  subjective  indicator  of  performance. 
Consequently  the  criterion  function  cannot  be  used  outside 
its  design  space  to  test  system  performance  and  is  therefore 
limited,  unless  the  criteria  and  their  parameters  are 
re-evaluated . 

It  may  seem,  that  after  careful  analysis  and  develop¬ 
ment  of  a  predictive  equation,  much  of  the  system’s  success 
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is  left  to  chance.  While  this  may  be  true,  the  designer 
will,  in  any  case,  have  greatly  reduced  the  possibilities 
for  failure  or  malfunction  and  will  be  better  equipped  to 
assess  problems  and  suggest  changes. 

Formal  Optimization 

This  section  is  the  final  step  in  choosing  an  optimal 
candidate  system,  one  that  will  satisfy  the  needs  and  goals 
identified  earlier.  The  candidate  system  chosen  yields  the 
highest  CFV  from  among  all  other  CFVs  considered.  Combining 
parameter  values  in  such  a  way  that  each  candidate  system 
receives  an  optimal  value  is  defined  by  Ostrofsky  as 
'optimization  within  a  candidate  system'  (14:134).  This  is 
done  to  assure  a  candidate  system  is  compared  in  its  'best 
light'  (14:134). 

The  computer  program  in  Appendix  F,  was  run  for  this 
optimization.  The  program  assembled  131,072  combinations  of 
parameter  values  and  compared  the  resulting  CFVs.  Five  CF 
values  were  recorded  to  show  any  variations  on  parameter 
values  and  the  resulting  CFV.  After  one  and  a  half  hours  of 
computation  time  the  top  five  candidate  systems  were 
outputted  and  appear  below  in  table  22. 

The  values  of  the  parameters  that  yielded  the  top  CFV 
were  then  used  to  evaluate  the  best  candidate  system  from 
amongst  the  activities  defined  in  the  feasibility  study. 
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Table  22.  Optimization  of  CFV 


Y(k) 

1st 

2nd 

3rd 

4th 

5th 

Y1 

3 

3 

3 

3 

3 

Y2 

5 

5 

5 

5 

5 

Y3 

30 

30 

30 

30 

30 

Y4 

10 

10 

10 

10 

10 

Y5 

13248 

13248 

13248 

13248 

13248 

Y6 

5000 

5000 

5000 

5000 

5000 

Y7 

455000 

455000 

455000 

455000 

455000 

Y8 

100000 

100000 

100000 

5000 

100000 

Y9 

400000 

400000 

400000 

400000 

400000 

YIO 

10000 

10000 

2000 

10000 

10000 

Y1 1 

3400 

500 

3400 

3400 

3400 

Y12 

1 

1 

1 

1 

1 

Y13 

1 

1 

1 

1 

1 

Y14 

1 

1 

1 

1 

1 

Y15 

5 

5 

5 

5 

5 

Y16 

16640 

16640 

16640 

16640 

16640 

Y17 

1 

1 

1 

1 

6 

CFVs 

. 906800 

.906258 

.905306 

.  889053 

.  873231 

The  highest  CFV  of  .9068  suggested  a  candidate  system  from 
Concept  III,  composed  of  activities  and  alternatives  A1 ,  B3 , 
Cl ,  D7 ,  E4 ,  F7 .  G7 ,  and  HI . 

Such  a  candidate  system  could  be  fulfilled  by  way  of 
implementing  a  centralized  computer  controlled  voice 
messaging  hardware  and  software.  The  system  would  be 
capable  of  collectively  routing  calls,  prioritizing  them, 
and  providing  requested  information.  Voice  messaging 
systems  rely  on  caller  Inputs  which  take  the  form  of 
Touchtone  (28)  telephone  signals.  The  system  could  work  any 
length  of  time  and  hold  several  callers  in  a  queue, 
periodically  informing  them  of  further  duration.  The  system 
could  be  programed  to  provide  requested  information 
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utilizing  a  hierarchial  decision  tree.  Routine  and  followup 
appointments  could  be  made  by  the  system  with  more  urgent 
and  acute  care  passed  on  to  actual  appointment  clerks. 

The  system  would  provide  a  very  useful  service  to  the 
medical  center  and  would  undoubtedly  relieve  much  of  the 
present  congestion.  Other  benefits  may  be  acquired  while 
using  such  an  automated  system.  Quinn  noted  that 
"technology,  properly  applied  can  enhance  productivity, 
quality  and  economic  output  in  the  services  sector" 

( 16 ; 50)  . 

This  concludes  the  formal  methodology  development  for 
this  study.  An  indepth  background  study  on  the  field  of 
voice  messaging  follows  to  further  extoll  the  virtues  and 
advantages  of  such  a  system.  Suffice  it  to  say,  the  chosen 
candidate  system,  comprised  of  an  automated  voice  messaging 
system,  will  satisfy  the  goals  and  needs  previously 
addressed . 

Summary 

This  Preliminary  Activities  section  defined  criteria 
that  would  quantifiably  assist  in  the  selection  of  the 
candidate  system.  Once  defined,  the  criteria  were  weighted 
as  to  their  importance.  This  was  done  to  identify  the  more 
important  criteria. 

In  order  to  measure  the  criteria,  parameters  were 
developed  and  were  used  to  quantify  the  criteria.  These 
parameters  were  combined  into  functions  and  then  acceptable 
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ranges  were  established  to  define  a  region  of  study. 
Utilizing  the  criteria  function,  parameter  ranges,  and 
weighted  values,  a  single  criterion  function  was  formed. 

From  this  equation  varying  parametric  values  were 
introduced  with  the  goal  of  achieving  the  highest  CFV. 

After  selecting  the  optimal  candidate  system,  any  revision 
or  change  to  the  criteria  should  occur  only  after  careful 
consideration  of  information  obtained  from  the  sensitivity, 
compatibility,  and  stability  analyses. 

The  further  aspect  of  system  implementation  is  dealt 
with  in  the  Detail  Activities  portion  of  the  Design 
Methodology  and  is  beyond  the  scope  of  this  research.  Such 
undertakings  should  be  handled  after  a  thorough  review  of 
system  capabilities.  Although  new  to  the  area  of  appoint¬ 
ment  systems,  automated  voice  messaging  is  a  viable 
alternative  to  current  clerk  assisted  appointment 
scheduling.  Acceptance  and  successful  implementation  of 
such  a  system  requires  an  understanding  of  what  voice 
messaging  is  and  how  it  may  be  integrated  into  the  field  of 
appointment  systems. 
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V.  Chosen  Candidate  System  Baokground 

Having  identlfed  a  voice  messaging  system  to  fulfill 
the  needs  outlined  previously,  it  becomes  necessary  to 
develop  a  general  comprehension  of  voice  messaging. 

Introduction 

Since  AT&T’s  conception  of  the  dual -tone-mul ti frequency 
technique,  the  Touchtone  system  has  become  an  industry 
standard  (18:120).  According  to  Doug  Mangin,  Communication 
Relations  Manager  for  Ohio  Bell  Communications,  more  than  82 
percent  of  the  residential  telephone  lines  in  the  greater 
Dayton  area  alone  currently  subscribe  to  Touchtone  service 
(11).  This  wide  spread  use  of  Touchtone  service  has  enabled 
the  use  of  telephones  as  remote  terminals  for  communication 
with  computers. 

Industry  Development 

In  1982  VMX  Inc.,  Dallas  TX,  originated  the  concept  of 
using  a  computer  to  digitally  record  and  redirect  voice 
messages  (8:c34).  Referred  to  as  voice  messaging,  the  heart 
of  the  system  entailed  a  computer  with  three  unique 
abilities: 

1.  The  ability  to  turn  the  sounds  carried  by  a 
telephone  wire  into  digital  data  files  that  can  be 
manipulated--copied ,  duplicated,  moved,  deleted-- 
as  any  other  computer  file. 

2.  The  ability  to  turn  those  files  back  into  sounds. 
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3.  The  ability  to  recognize  and  respond  to  telephone 

Touchtones.  (8;c34) 

One  of  the  leading  manufacturers  of  voice  recognition 
computer  hardware  is  VOTAN  of  Fremont  California.  Bruce 
Ryon ,  marketing  communications  manager  at  VOTAN,  describes 
voice  processing  as:  the  use  of  human  voice  to  communicate 
with  computers.  With  it,  people  can  access  and  use 
computers  over  the  phone,  and  in  areas  or  situations  where 
communicating  via  a  keyboard  is  not  feasible.  The  outgrowth 
of  the  voice  processing  technology  has  generally  been 
classified  into  four  modes:  "automated  attendant,  telephone 

answering,  information  center,  and  voice  mail"  (20:16). 

The  switchboard  operator  has  been  replaced  by  the  auto¬ 
mated  telephone-attendant.  This  system  responds  to  incoming 
calls  with  a  digitized  voice  message  asking  the  caller  to 
whom  they  wish  to  speak  and  subsequently  routing  their  call 
to  that  person’s  phone.  If  there  is  no  answer,  the  caller 
is  invited  to  leave  a  message  (also  digitally  recorded)  and 
review  it  before  saving  (8:c35).  Unlike  the  old  desk  top 
answering  machines,  voice  messaging  systems  are  not  on  an 
individual’s  desk,  but  Instead  are  centrally  located  for  use 
by  an  entire  company.  Thus  a  single  message  can  be  sent  to 
many  individuals  without  having  to  repeat  it  (2:40). 

Aside  from  the  simplistic  task  of  recording  messages, 
many  unique  applications  are  possible  using  the  Touchtone 
keypad  as  a  data  input  terminal.  One  of  the  first 
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Industries  to  realize  this  innovative  potential  was  the 


banking  industry. 

Industry  Applications 

Far  West  Federal  Savings  Bank  in  Portland  OR,  was 
handling  approximately  20,000  customer  telephone  inquiries  a 
month.  As  the  need  arose  to  automate  their  routine  calls, 
they  installed  INFOBOT,  "a  modular  voice  response  system 
from  Phoenix-based  Syntellect  Inc"  (21:84).  Customer  access 
began  in  1987  and  the  system  handles  15-20  percent  of  the 
Incoming  customer  service  calls.  These  calls  range  from 
account  balances  to  interest  earned  and  cleared  checks. 
Customer  response  has  been  very  favorable  and  the  system  has 
been  both  economical  and  efficient  (21:85). 

Another  early  use  of  Touohtone  banking  was  by  the 
Fulton  Federal  Savinga  and  Loan  of  Atlanta  QA.  Their 
system,  activated  in  July  1987,  allows  customers  to  pay 
their  bills,  verify  account  balances,  verify  withdrawals  and 
deposits,  and  inquire  if  specific  checks  have  cleared. 

Fulton  employees  provide  more  informed  answers  with  the 
system,  generating  better  customer  service.  The  two  biggest 
advantages  to  such  a  system  are  convenience  to  the  customer 
and  control  by  programmers  (15:108). 

Recently,  Wright-Patt  Credit  Union  in  Fairborn  OH,  also 
installed  a  telephone  automation  system  for  its  customers. 
This  automated  attendant,  referred  to  as  CALL-24,  handles 
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approximately  20,000  calls  a  month,  offering  a  range  of 
account  inquiries  and  money  transfer  between  accounts.  Less 
than  a  year  old  the  system  has  reduced  customer  service 
calls  handled  by  account  representati ves  and  has  been 
considered  a  tremendous  success  by  the  organization  (13). 
Telephone  automation  is  also  being  used  as  a  tool  for  regis¬ 
tration  and  scheduling  of  college  courses.  In  addition  to 
successful  registration  systems  at  Brigham  Young  University, 
Georgia  State  University,  and  Ohio  State  University,  Rio 
Salado  Community  College  in  Phoenix  AZ  installed  a  system  in 
July  1985.  The  system  makes  registration  easily  accessible 
for  a  scattered  community  college  population,  making  it 
quick,  simple,  and  convenient.  Potential  savings  include 
the  retention  of  possible  students  because  of  the  conveni¬ 
ence.  Future  applications  include  bilingual  responses, 
"tuition  calculation,  recording  fee  payments  by  charge  card 
or  bank  draft  and  student  account  fee  balance"  (17:46). 

Medical  Service  Use 

The  wide  range  of  use  for  voice  technology  has  encour¬ 
aged  its  growth  in  many  Industries.  Hospital  applications 
have  been  developed  but  do  not  yet  enjoy  widespread  use. 
Informal  telephone  inquiries  to  five  local  area  hospitals, 
in  the  greater  Dayton  OH  area,  revealed  a  common  practice 
of  manual  appointment  scheduling.  This  is  unfortunate  since 
voice  processing  provides  a  faster,  more  natural,  and  in 
many  cases  a  more  cost-effective  means  of  interacting  with 
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the  automated  equipment  common  in  the  healthcare  industry 
(19:33)  . 

The  limited  use  of  voice  messaging  in  hospitals  has 

been  restricted  primarily  to  multifaceted  medical  centers, 

usually  military  facilities.  In  order  to  gain  a  better 

understanding  of  voice  messaging  features  a  communications 

vendor  specializing  in  voice  technology  was  selected. 

Microlog  Corporation  of  Germantown  MD  was  chosen  based  on 

its  product  availability  in  the  QSA  catalog,  experience  with 

military  medical  facilities  and  unique  product  features. 

This  study  in  no  way  endorses  Microlog  or  makes  any  claims 

regarding  their  product  or  services. 

One  of  Microlog's  primary  voice  messaging  systems  is 

the  VolceConnect  System  (VCS)  3500.  John  Corrigan, 

Microlog’s  Major  Account  Executive,  extolled  many  of  the  VCS 

3500’s  unique  feature.  Some  of  these  attributes  include: 

Outbound  dialing  -  Useful  for  patient  reminders, 
notifying  about  test  results,  or  a  general  recall. 

Posi tive  Voice  Response  -  Ability  to  handle  calls 
originating  from  either  rotary  or  Touchtone  phones. 

Virtual  Uni imi ted  Message  Length  -  Limitation  is  by 
virtue  of  disk  storage  capacity. 

Remote  Message  Changes  -  Stored  messages  and  prompts 
can  be  altered  by  authorized  users,  making  emergency 
updates  possible. 

Touch  Tone  Digit  Translation  -  Convert  Touchtone 
signals  into  verbal  descriptions  such  as  a  person’s 
name  or  department. 

Call  Queuing  -  System  is  able  to  inform  patients  of 
their  status  while  on  hold,  giving  them  their  position 
in  line. 
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Independent  Line  Operation  -  Each  phone  line  can  be 
dedicated  for  a  unique  purpose  within  the  system. 

Independent  Line  Application  -  Different  applications 
can  be  combined  on  a  single  line  such  as  pharmacy 
refills  and  appointments.  (29:4) 

The  features  of  the  VCS  3000  and  other  competitive  systems, 

show  a  strong  potential  to  satisfying  the  expectations  and 

needs  of  an  appointment  service.  Mlcrolog  currently  has  a 

VCS  system  In  place  at  both  Walter  Reed  Army  Medical  Center 

and  Bethesde  Naval  Hospital  (4).  These  high  volume  military 

treatment  facilities  are  undoubtedly  beneflttlng  from  the 

services  of  a  voice  messaging  system.  Additionally,  the 

Medical  Center  at  Langley  AFB  VA  Is  contemplating  the 

acquisition  of  the  VCS  system  establishing  the  product  as  a 

well  known  commodity  within  the  military  and  medical  fields. 

Although  Microlog’s  products  are  well  known  and  sport 

many  useful  features,  other  systems  exist  that  may  be 

equally  satisfying  to  the  needs  of  an  appointment  system. 

Only  adequate  research  and  cost/benefit  analysis  will 

extract  a  system  most  useful  to  a  particular  medical  center. 

Voice  messaging  systems  are  being  used  in  other  ways 

outside  of  appolnt'dnt  systems.  St.  Joseph’s  Hospital  In 

Denver  CO,  automated  their  meal  preparation  with  voice 

processing.  Significant  Increases  In  efficiency  and  the 

elimination  of  errors  has  been  observed  (19:34).  Kalser- 

Permanente  Health  Foundation  installed  a  voice  recognition 

and  output  system  In  1986  In  one  southern  California 

facility,  to  remind  hundreds  of  patients  of  their  appolnt- 
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ments  each  day.  After  completing  the  calls,  ’the  system 
produces  a  list  of  people  contacted,  noting  which  of  them 
can  make  their  appointments  and  which  will  need 
rescheduling’  (19:33,34). 

Conclusion 

The  IQSO's  may  be  considered  the  efficiency  through 
automation  decade.  The  ability  to  boost  productivity  and 
efficiency  through  voice  technology  gives  hospitals  and 
other  service  related  industries  the  ability  to  reduce 
errors  and  Improve  customer  satisf action .  Several  major 
military  medical  facilities  are  utilizing  Microlog’s  VCS 
series  automated  voice  messaging  systems  in  their 
appointment  system.  Aside  from  its  many  unique  features  the 
VCS  3000  improves  a  hospitals  call  answering  ability  and  can 
sort  calls  expedlously.  Continued  research  and  development 
will  make  automated  appointment  scheduling  a  reality  for  the 
hospitals  of  tomorrow. 
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VI  .  Concluaiona  and  Reconunendatlons 


This  study  was  conducted  under  the  auspices  of 
improving  access  to  the  appointment  desk  at  the  USAF  Medical 
Center  Wright-Patterson  AFB  OH.  The  literature  suggested  a 
need  to  Improve  appointment  systems  overall.  Little  was 
found,  however,  that  dealt  directly  with  the  problem  of 
appointment  system  access. 

The  methodology  used  was  both  an  engineering-design 
tool  and  an  instructional  aid.  In  performing  the  latter  the 
methodology  soon  became  an  exercise  in  systematically 
thinking  through  the  problem.  The  iterative  process  never 
claimed  an  optimum  output,  but  instead  led  the  designer  to  a 
'best*  or  optimal  solution. 

To  arrive  at  this  solution  required  an  assessment  of 
need,  organizational  goals,  limits,  and  boundaries.  The 
method  then  guided  the  designer  to  a  quantifiable  approach 
to  the  solution.  This  allowed  the  choice  of  an  optimal 
system  from  amongst  others  based  on  a  cardinal  scale.  The 
criterion  function  values  stemmed  from  descriptive  criteria, 
their  weighted  importance,  and  expected  criteria  behavior  or 
functions . 

Consideration  was  given  to  the  sensitivity  effect 
certain  parameters  have  on  the  CFV.  Based  on  information 
obtained  from  the  sensitivity,  compatibility,  and  stability 
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analyses,  recommendations  were  made  as  to  the  magnitude  of 
change  on  the  CFV  from  altering  parameter  values. 

The  proposed  candidate  system,  featured  a  voice 
messaging  system  to  solve  the  needs  of  the  Medical  Center’s 
appointment  system.  Voice  messaging  has  been  applied  in 
numerous  endeavors  Including  appointment  systems  as 
demonstrated  by  the  Microlog  Corporation  product. 

Conclusions 

The  information  gathered  from  the  questionnaires 
revealed  an  appointment  system  that  did  not  perform 
satisfactorily  for  the  Medical  Center  staff.  Some  comments 
were  very  useful  and  indicated  specific  areas  of  concern. 

The  methodology  resulted  in  a  number  of  criteria  and 
associated  parameters  that  were  used  to  quantify  the  ‘best’ 
solution.  The  parameters  themselves  were  sometimes 
difficult  to  measure  and  no  attempt  was  made  to  measure 
the  more  abstract  ones.  Likewise,  some  parameter  minimums 
and  maximums  were  also  difficult  to  establish.  Careful 
thought  had  to  be  given  for  what  was  desired  to  be  minimized 
and  what  was  needed  to  be  maximized.  The  criterion  function 
provided  a  quantifiable  way  to  input  system  parameters  and 
relate  them  together  in  an  effort  to  rank  each  candidate 
system. 

Further  research  into  appointment  systems,  indeed  of 
all  medical  service  communications,  should  reveal  that 
automated  telephone  attendants  are  but  temporary  solutions. 
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Complete  interconnectivity  and  ultimate  customer  satisfac¬ 
tion  will  only  come  through  planning  and  an  intelligent 
interface.  Artificial  intelligence  may  be  the  necessary 
ingredient  to  solve  the  long  range  communications  plans  of  a 
multifaceted  health  care  organization. 

Limitations 

Parameter  ranges  played  a  significant  part  in  the 
development  of  the  criterion  function.  Unfortunately,  time 
constraints  prevented  the  accumulation  of  more  precise 
values.  The  information  gathered,  however,  does  provide  a 
significant  look  at  an  acceptable  range  of  values  for  the 
parameters  considered. 

While  the  sensitivity  analysis  did  produce  some  insight 
into  parameter  effects  on  the  CFV,  it  was  limited  to  varying 
only  one  parameter  at  a  time.  A  more  in-depth  and  improved 
simulation  w<^uld  be  able  to  vary  each  parameter  successively 
against  each  other  in  a  17  dimensional  space.  This  would, 
in  effect,  vary  first  one,  then  two,  and  eventually  all 
17  parameters  over  a  range  of  values  and  measures  the 
sensitivity  of  change  on  the  CFV.  Such  information  would 
prove  useful  as  the  more  detail  a  designer  possesses  the 
more  likely  he  is  to  develop  a  system  that  will  fit  together 
smoothly . 

After  completing  the  criterion  function,  the  objective 
was  to  find  a  combination  of  parameter  values  that  yielded 

I 
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the  highest  CFV.  Initially  a  computer  program  was  designed 
to  evaluate,  where  possible,  up  to  10  values  for  each 
parameter.  This  would  have  required  an  evaluation  of 
3.84  X  10^*  combinations.  To  complete  such  an  analysis 
would  have  required  a  computer  to  solve  and  evaluate  the 
criterion  function  1,217,656  times  per  second  for  an  entire 
year.  This  computational  limitation  reduced  the  number  of 
parameter  values  to  just  two  each,  but  still  resulted  in 
more  than  130,000  combinations. 

Recommendations 

With  a  growing,  more  demanding  patient  population,  it 
becomes  imperative  to  provide  adequate  means  for  them  to 
access  the  Medical  Center’s  appointment  system.  At  the 
present,  implementation  of  automated  systems  is  the  most 
practical  solution.  Information  given  in  the  Needs 
analysis,  points  to  the  need  for  automating  access  to  the 
appointment  system.  Linking  an  automated  voice  messaging 
system  to  the  present  patient  database  would  eliminate 
congestion  and  provide  a  more  acceptable  entryway  to  the 
Medical  Center’s  appointment  system. 

The  need  exists  to  further  study  and  evaluate  the 
remaining  portion  of  the  appointment  system,  where  this 
study  left  off.  Specifically  the  appointment  system  should 
be  examined  for  enhancements  and  improvements  once  or. 
appointment  is  scheduled.  Questions  two,  three,  and  four  of 
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the  Questionnaire  specifically  address  this  area  of  study 
and  may  be  used  for  an  initial  problem  analysis. 

Additional  work  remains  to  investigate  the  available 
voice  messaging  systems  on  the  market  and  provide  an 
evaluation  of  the  most  promising.  The  result  of  this  could 
then  be  used  for  an  eventual  system  implementation. 
Ostrofsky’s  design  methodology  contained  a  fourth  section, 
‘Detailed  Activities,'  which  dictate  the  steps  necessary  to 
implement  a  specifically  chosen  system.  If  a  specific 
system  is  accepted,  then  adherence  to  Ostrofsky’s  Detailed 
Activities  section  could  successfully  implement  the  system. 

Replication  of  this  research  may  be  able  to  redefine 
the  criteria  and  associated  parameters  and  in  so  doing 
develop  a  more  concise  criterion  function.  At  the  same  time 
consideration  might  also  be  given  to  further  analysis  of 
parameter  and  criterion  interactions.  Such  research  would 
enhance  the  validity  of  this  study  and  provide  a  more 
detailed  analysis  of  the  problem. 

Summary 

The  selection  of  an  optimal  candidate  system  was  a 
multi-step  process.  Each  succeeding  step  built  on  the  one 
preceding  it,  channeling  information  and  specifics  for 
evaluation  and  development.  The  resulting  criterion 
function  provided  a  quantifiable  means  to  select  a  "best’ 
candidate  system. 
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While  the  need  to  improve  the  appointment  system  by  way 
of  an  automated  voice  messaging  system  was  substantiated, 
the  long-term  solution  involves  utilization  of  artificial 
intelligence  programs. 

While  the  limitations  discussed  prevented  the  develop¬ 
ment  of  a  more  precise  criterion  function,  it  was  clear  that 
what  did  develop  provided  a  quantifiable  means  to  select  a 
single  candidate  system  to  meet  the  needs  of  the  Medical 
Center’s  appointment  system. 

After  careful  evaluation  of  existing  models  it  is 
recommended  that  a  voice  messaging  system  be  implemented. 
Additional  research  remains  to  evaluate  the  remaining 
portion  of  the  appointment  system  to  enhance  its 
operability.  Replication  of  this  study  may  provide  more 
precise  criteria  and  parameters  and  further  refine  the 
criterion  function. 

The  complexity  of  appointment  systems  is  a  continuously 
changing  problem.  In  a  non-profit  service  industry, 
customer  satisfaction  becomes  a  measure  of  success.  For  the 
military  professional  the  availability  and  dependable 
accessibility  of  competent  medical  care  is  necessity. 
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Appendix  A: 


USAF  Medical  Center  Ouestionnaire 


Wright -Pat ter son  AFB 
USAF  Medical  Center  Appointment  System 
Staff  Interview  Survey  Questionnaire 


PURPOSE :  This  study  is  being  conducted  to  determine  how  you 

feel  about  the  system  used  for  making  outpatient  appointments 
at  the  medical  center.  It  should  take  you  approximately  5 
minutes  to  complete  this  survey.  The  results  of  this  survey 
may  have  a  significant  impact  on  future  policy  and  guidance 
of  the  appointment  system. 

All  answers  to  the  questions  should  be  based  on  your 
experience  with  the  appointment  system  at  the  Medical  Center 
and  not  any  other  experience  you  may  have  had  with  other 
appointment  systems.  Your  answers  will  be  combined  with 
those  of  other  staff  members,  and  presented  for  analysis  at 
the  completion  of  the  survey. 

Your  cooperation  in  completing  this  questionnaire  will  be 
greatly  appreciated  and  will  provide  valuable  information 
which  may  be  used  to  make  the  outpatient  appointment  system 
serve  you  better. 


CHARLES  H.  ROADMAN  II 
Commander 


Col , 


USAF,  MC 
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1.  Which  statement  best  describes  your  current  status: 

(  )  Executive  Mgmt  (  )  Department  Chair 
(  )  Supervisor  (  )  Appointment  Clerk 

Please  circle  the  number  which  best  describes  your  feelings 
about  each  of  the  following  issues  related  to  the  way  in 
which  the  appointment  system  operates.  High  numbers  indi¬ 
cate  satisfaction  and  lower  numbers  indicate  dissatisfac¬ 
tion.  Please  consider  only  the  USAF  Medical  Center,  Wright- 
Patterson  AFB  appointment  system  and  not  any  other  system 
you  have  had  experience  with. 

Very  Very  Dis¬ 

satisfied  Satisfied  Neither  Dissatisfied  satisfied  N/A 

5  4  3  2  1  0 


2.  Your  ability  to  obtain  patient  and  schedule  information 
from  the  appointment  system. 

5  4  3  2  1  0 

3.  How  satisfied  are  you  that  the  appointment  clerk  matches 
the  patient  with  the  proper  appointment  slot? 

5  4  3  2  1  0 

4.  How  satisfied  are  you  with  being  able  to  contact  the 
appointment  clerk  when  you  need  to? 

5  4  3  2  1  0 

5.  How  satisfied  are  you  with  the  number  of  appointment 
scheduling  people  in  your  area? 

5  4  3  2  1  0 

6.  What  is  your  overall  opinion  of  the  appointment  system 
used  to  make  appointments? 

5  4  3  2  1  0 

7.  If  you  could  change  anything  about  the  appointment 
system,  what  would  it  be? 


THANK  YOU  FOR  YOUR  TIME  AND  COOPERATION 
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Appendix  B.  Stratified  Tabulated  Questionnaire  Results 

Response  range:  0  (N/A)  to  5  (Very  Satisfied). 

2.  Your  ability  to  obtain  patient  and  schedule  information 
from  the  appointment  system. 


N/A 

V/D 

D 

N/S/D 

S 

V/S 

0 

1 

2 

3 

4 

5 

Executive  Management  (9) 

33% 

0% 

11% 

11% 

33% 

11% 

Department  Chair  (7) 

0% 

0% 

14% 

14% 

57% 

14% 

Subtotal  (16) 

19% 

0% 

13% 

13% 

44% 

13% 

Supervisor  (7) 

0% 

0% 

14% 

14% 

29% 

43% 

Appointment  Clerk  (11) 

0% 

0% 

9% 

9% 

36% 

45% 

Subtotal  (18) 

0% 

0% 

1 1% 

11% 

33% 

44% 

COMBINED  TOTAL  (34) 

9% 

0% 

12% 

12% 

38% 

29% 

How  satisfied  are 
the  patient  with 

you  that 
the  proper 

the  appointment  clerk  matche 
appointment  slot? 

N/A  V/D  D  N/S/D  S  V/S 

0  1  2  3  4  5 

Executive  Management 

33% 

0% 

22% 

11% 

22% 

11% 

Department  Chair 

0% 

0% 

14% 

29% 

57% 

0% 

Subtotal 

19% 

0% 

19% 

19% 

38% 

6% 

Supervisor 

0% 

0% 

0% 

0% 

71% 

29% 

Appointment  Clerk 

9% 

0% 

0% 

18% 

27% 

45% 

Subtotal 

6% 

0% 

0% 

1 1% 

44% 

39% 

COMBINED  TOTAL 

12% 

0% 

9% 

15% 

41% 

24% 

How  satisfied  are  you 

with 

being 

able 

to 

contact 

the  appointment  clerk 

when 

you  need  to? 

N/A 

V/D 

D 

N/S/D 

S 

V/S 

0 

1 

2 

3 

4 

5 

Executive  Management 

33% 

0% 

1 1% 

22% 

33% 

0% 

Department  Chair 

0% 

0% 

14% 

29% 

29% 

29% 

Subtotal 

19% 

0% 

13% 

25% 

31% 

13% 

Supervisor 

0% 

0% 

0% 

0% 

14% 

86% 

Appointment  Clerk 

73% 

0% 

0% 

0% 

18% 

9% 

Subtotal 

44% 

0% 

0% 

0% 

17% 

39% 

COMBINED  TOTAL 

32% 

0% 

6% 

12% 

24% 

26% 
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5.  How  satisfied  are  you  with  the  number  of  appointment 
scheduling  people  in  your  area? 


N/A 

V/D 

D 

N/S/D 

S 

V/S 

0 

1 

2 

3 

4 

5 

Executive  Management 

44% 

33% 

0% 

11% 

11% 

0% 

Department  Chair 

0% 

29% 

71% 

0% 

0% 

0% 

Subtotal 

25% 

31% 

31% 

6% 

6% 

0% 

Supervisor 

0% 

14% 

14% 

14% 

43% 

14% 

Appointment  Clerk 

0% 

18% 

9% 

0% 

45% 

27% 

Subtotal 

0% 

17% 

11% 

6% 

44% 

22% 

COMBINED  TOTAL 

12% 

24% 

21% 

6% 

26% 

12% 

What  is  your  overall  opinion 

of 

the  appointment 

system 

used  to  make  appointments? 

N/A 

V/D 

D 

N/S/D 

S 

V/S 

0 

1 

2 

3 

4 

5 

Executive  Management 

33% 

1 1% 

22% 

0% 

33% 

0% 

Department  Chair 

0% 

0% 

43% 

43% 

14% 

0% 

Subtotal 

19% 

6% 

31% 

19% 

25% 

0% 

Supervisor 

0% 

0% 

29% 

14% 

57% 

0% 

Appointment  Clerk 

0% 

9% 

9% 

9% 

64% 

9% 

Subtotal 

0% 

6% 

17% 

11% 

61% 

6% 

COMBINED  TOTAL 

9% 

6% 

24% 

15% 

44% 

3% 

7,  If  you  could  change  anything  about  the  appointment 
system,  what  would  it  be? 

Executive  Management: 

1.  Expand  access,  that  is  add  more  clerks. 

2.  Increase  training  for  clerks. 

3.  Patients  have  difficulty  phoning  into  the  system  and 
have  little  confidence  in  the  computer. 

4.  Make  some  change  to  the  waiting  list. 

5.  Number  of  phone  lines 

6.  Number  of  appointment  clerks. 

Department  Chairmen: 

1.  Control  over  pre-recorded  messages  heard  by  patients 
when  calling  the  appointment  desk. 

2.  Patients  need  to  be  made  aware  of  Information  phone 
number . 

3.  Improved  access  to  the  system,  to  include  additional 
staff  to  meet  peak  demand  times. 

4.  More  phone  lines. 
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Department  Chairmen  (continued) : 

5.  More  appointment  clerks. 

6.  Upgrade  appointment  clerk  positions. 

7.  More  medically  qualified  appointment  managers. 

8.  Increase  manning  to  allow  better  rapport  with  clerk 
and  patient. 

Supervisors : 

1.  Enough  help  to  cover  the  busy  times. 

2.  Hire  clerks  at  QS-4  rating. 

3.  Total  decentralization  of  appointment  system. 

4.  More  personnel  to  meet  patient  needs  better. 

5.  Complaints  received  concerning  patients  being  unable 
to  get  through  busy  phone  lines. 

6.  Difficult  for  patients  to  make  same  day  appointments. 

7.  Computers  need  to  respond  faster 

8.  Another  clerk  on  busy  days  would  be  helpful. 

9.  Toll  free  1-800  number  would  be  beneficial  to  patients 
calling  long  distance. 

10.  Patients  wait  too  long  on  system,  should  either  get  an 
answer  or  a  busy  signal. 

11.  Something  indication  of  what  clinic  is  reached  when  put 
on  hold. 

12.  Informational  response  to  indicate  best  times  to  try 
calling  back. 

13.  More  flexibility,  to  allow  providers  to  change  or  add 
appointment  times. 

Appointment  Clerks; 

1.  More  available  appointments  without  overbooking. 

2.  Appointment  times  need  to  be  better  aligned  with 
problem  type. 

3.  Allow  providers  to  make  their  own  follow  up 
appointments  for  patients. 

4.  More  appointment  clerks. 

5.  Ability  to  make  multiple  appointments  for  a  single 
patient . 

6.  Put  a  quite  ringer  on  the  telephones. 

7.  Hire  temporary  appointment  clerks. 

8.  Closer  coordination  la  needed  between  clinics  and 
appointment  clerks.  Scheduling  changes  made  without 
notifying  clerks. 

9.  Keep  clerks  current  on  all  new  appointment  procedures. 

10.  Slow  computer  responses,  due  to  overloading. 

11.  Better  telephone  equipment  needed. 

12.  Diversity,  some  kind  of  break  is  needed  or  change  of 
pace . 

13.  Limit  the  number  of  information  calls. 
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Appendix  E;  Source  Code  for  Sinai e  Value 
Sensitivity  Analysis 


10 - 

20  REM  SOURCE  CODE,  VERSION  1.1 
30  REM  SENSITIVITY  ANALYSIS 
40  REM  FOR  GRADUATE  THESIS 
50  REM 

60  REM  WRITTEN  IN  TURBO  BASIC  XL  VER  1.5 
70  REM  (C)  1985  BY  FRANK  OSTROWSKI 

30 - 

90  REM 

100  DIM  Y(20)  ,OLDY<20) 

110  LPRINT  SENSITIVITY  ANALYSIS  AT  5%  ****" 

120  FOR  1=1  TO  17 

130  Y(l)=l  .5:Y(2)=3:Y(3)=15:Y(4)=5:Y(5)=12721 
14  0  Y  <6)  =2750  ;  Y  (7)  =252500 :  Y  (8)  =52500  :  Y  (9)  =225000  : 

Y  (10)  =6000 

150  YC11)=1950:Y(12)=0.5  :Y(13)=0.5  :Y(14)=0.5  :Y(15)=12 .5  ; 

Y(16)=8560:YC17)=3.5 
160  GOSUB  320 
170  OLDYCI)=Y(I) 

180  Y<I)=Y(I)  *1 .05 

190  OLDCF=CF 

200  GOSUB  320 

210  CFDIFF=CF-OLDCF 

220  CHANGE= (CFDIFF/OLDCF) *100 

230  OLDCF=INT(OLDCF*10000)/10000 

240  OLDY (I) =INT (OLDY (I) *10000) /lOOOO 

250  Y(I)  =INT(Y(I)  *1000)/1000 

260  CF=INT(CF*10000)/10000 

270  CFDIFF=INT(CFDIFF*10000)/10000 

280  CHANGE=INT (CHANGE*100)/100 

290  LPRINT  "Y“;I .OLDCF, OLDY (I) ,Y(I) ,CF,CFDIFF, CHANGE 
300  NEXT  I 
310  END 

320  CF1  =  0.24*  ((Y<1)*Y(2))/15)+0.19*  (((-0.5*  (Y(3)  *Y(4))  ^2) 
+50)/-127058) 

330  CF2=CFl  +  0 . 16*  (  (  (Y  (5)  +Y  (6)  +Y  (7)  +Y  (8)  +Y  (9)  +Y  (10)  + 

Y  (ID) -175195)/ 856453) 

340  CF=CF2+0.22*  (((Y(12)+Y(13)+Y(14))/Y(15))/0.6)+0.19* 

(  (  (Y  (16)  /  (Y  (16)  +Y  (17)  )  )  -0 . 9979)  / 1 .7E-03) 

350  RETURN 
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Appendix  F.  Source  Code  for  Multiple  Values 
in  a  Sensitivity  Analysis 


10 - 

20  REM  SOURCE  CODE,  VERSION  2.0 
30  REM  MULTIPLE  PERCENTAGE  RUN 
40  REM  SENSITIVITY  ANALYSIS 
50  REM  FOR  GRADUATE  THESIS 
60  REM 

70  REM  WRITTEN  IN  TURBO  BASIC  XL  VER  1,5 
80  REM  (C)  1985  BY  FRANK  OSTROWSKI 

90 - 

100  LPRINT  "♦***  SENSITIVITY  ANALYSIS 
110  DIM  Y(20)  ,OLDY(20) 

120  FOR  1=1  TO  17 
130  GOSUB  500 

140  GOSUB  460 

150  LPRINT  '‘Y(";I;")“,"MEAN  VALUE  =  ";Y(I) 

160  LPRINT  "INITIAL  CF  VALUE  =  ";CF: LPRINT 
170  LPRINT  ; LPRINT  "Yj  +  %Yj  CFV  CHG .  CFV  %CHG" 

180  PCT=1.05 

190  OLDY(I)=Y(I) 

200  FOR  LOOP=l  TO  5 
210  GOSUB  500 

220  GOSUB  460 

230  OLDY(I)=OLDY(I)*PCT 

240  Y(I)=OLDY(I) 

250  OLDCF=CF 

260  GOSUB  460 

270  DIFFCF=CF-OLDCF 

280  CHANGE= (DIFFCF/OLDCF) *100 

290  OLDCF=INT(OLDCF*10000)/10000 

300  OLDY  Cl) =INT (OLDY (I) *10000) /lOOOO 

310  Y(I)=INT(Y(I)  *10000)710000 

320  CF=INT<CF*10000) 710000 

330  DIFFCF=INT (DIFFCF*10000) 710000 

340  IF  YCI)>100  THEN  Y  (I)  =INT  (Y  Cl)  ) 

350  CHANGE=INTCCHANGE*100)7100 

360  LPRINT  Y CD .CF.DIFFCF, CHANGE 

370  NEXT  LOOP 

380  IF  PCT=0.95  THEN  410 

390  PCT= 0. 95 : GOSUB  500 

400  GOTO  190 

410  LPRINT  ; LPRINT 

420  IF  I73=INTCI73)  THEN  LPRINT  CHR$ C12) s LPRINT 
"*****  SENSITIVITY  ANALYSIS  **♦**" 

430  NEXT  I 
440  END 

450  - 
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460  CF1=0 .24*  (  (Y  (1)  *Y  (2)  )  /15)  +0. 19*  C  (  (-0.5*  (Y  (3)  *Y  (4)  )  ''2) 


+50)/-127058) 

470  CF2=CFl  +  0 . 16*  (  (  (Y  (5) +Y  (6) +Y  (7) +Y  (8) +Y  (9) +Y  (10) +Y  (11)  ) 
-175195)7856453) 

4  80  CF=CF2  +  0 .22*  (  (  (Y  (12) +Y  (13)  +Y  (14)  )  /Y  (15)  )  70 . 6)  +0 . 19* 

(  (  (Y  (16)7  (Y  (16)  +Y  (17)))-0.9979)71  .7E-03) 

490  RETURN 

i?8  ? ^VSHliiooo-.Yaoy >6ooo 


520  Y(11)=1950:Y(12)=0.5:Y(13)=0,5:Y(14)=0.5;Y(15)=12 .5: 

Y(16)=8560:Y(17)=3.5 
530  RETURN 
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Appendix  G:  Computer  Output  of  Multiple 

Values  on  a  Sensitivity  Analysis 


****  SENSITIVITY 

ANALYSIS  **** 

Y(l) 

INITIAL 

MEAN  VALUE  =1.5 

CF  VALUE  =  0.3798164157 

Yj  +  SfeYj 

CFV 

CHG.  CFV 

%CHG 

1.575 

0.3834 

3.6E-03 

0.94 

1.6537 

0.3871 

7.3E-03 

1.94 

1.7363 

0.3911 

0.0113 

2.98 

1.8231 

0.3953 

0.0155 

4.08 

1.9142 

0.3997 

0.0198 

5.23 

1  .425 

0.3762 

-3.6E-03 

-0.95 

1 . 35  37 

0.3727 

-7.1E-03 

-1.85 

1.286 

0.3695 

-0.0103 

-2.71 

1  .2217 

0.3664 

-0.0134 

-3.52 

1  . 1606 

0.3635 

-0.0163 

-4.29 

Y(2) 

INITIAL 

MEAN  VALUE  =  3 

CF  VALUE  =  0.3798164157 

Yj  +  %Yj 

CFV 

CHG.  CFV 

%CHG 

3.15 

0.3834 

3.6E-03 

0.94 

3 . 3075 

0.3871 

7.3E-03 

1.94 

3.4728 

0.3911 

0.0113 

2.98 

3.6464 

0.3953 

0.0155 

4.08 

3.8287 

0.3997 

0.0198 

5.23 

2.85 

0.3762 

-3.6E-03 

-0,95 

2.7075 

0.3727 

-7.1E-03 

-1.85 

2.5721 

0.3695 

-0.0103 

-2.71 

2.4434 

0.3664 

-0.0134 

-3.52 

2.3212 

0.3635 

-0.0163 

-4  .29 
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*****  SENSITIVITY  ANALYSIS  ***** 
Y(3)  MEAN  VALUE  =  15 

INITIAL  CF  VALUE  =  0.3798164157 


Yj  +  %Yj 

CFV 

CHG.  CFV 

%CHG 

15.75 

0.3802 

4E-04 

0.11 

16.5375 

0.3807 

9E-04 

0.23 

17.3643 

0.3812 

1 .4E-03 

0.37 

18.2325 

0.3818 

2.0E-03 

0.52 

19.1441 

0.3824 

2.6E-03 

0.69 

14  .25 

0.3794 

-5E-04 

-0.11 

13.5375 

0.379 

-8E-04 

-0.21 

12.8606 

0.3787 

-1 .2E-03 

-0.3 

12.2175 

0.3784 

-1 .5E-03 

-0.38 

11.6066 

0.3781 

-1 .7E-03 

-0.45 

YC4) 

INITIAL 

MEAN  VALUE  =  5 

CF  VALUE  =  0.3798164157 

Yj  +  %Yj 

CFV 

CHG.  CFV 

%CHG 

5.25 

0.3802 

4E-04 

0.11 

5.5125 

0.3807 

9E-04 

0.23 

5.7881 

0.3812 

1.4E-03 

0.37 

6.0775 

0.3818 

2.0E-03 

0.52 

6.3813 

0.3824 

2.6E-03 

0.69 

4.75 

0.3794 

-5E-04 

-0.11 

4.5125 

0.379 

-8E-04 

-0.21 

4.2868 

0.3787 

-1 .2E-03 

-0.3 

4.0724 

0.3784 

-1  .5E-03 

-0.38 

3.8687 

0.3781 

-1 .7E-03 

-0.45 

Y(5) 

INITIAL 

MEAN  VALUE  =  12721 

CF  VALUE  =  0.3798164157 

Yj  +  %Yj 

CFV 

CHG.  CFV 

%CHG 

13357 

0.3799 

IE- 04 

0.03 

14024 

0.38 

2E-04 

0.06 

14726 

0.3801 

3E-04 

0.09 

15462 

0.3803 

5E-04 

0.13 

16235 

0.3804 

6E-04 

0.17 

12084 

0.3796 

-2E-04 

-0.04 

11480 

0.3795 

-3E-04 

-0.07 

10906 

0.3794 

-4E-04 

-0.09 

10361 

0.3793 

-5E-04 

-0.12 

984  3 

0.3792 

-6E-04 

-0.15 
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*****  SENSITIVITY 

ANALYSIS  ♦♦*** 

Y(6) 

INITIAL 

MEAN  VALUE  =  2750 

CF  VALUE  =  0.3798164157 

Yj  +  %Y3 

CFV 

CHG.  CFV 

%CHG 

2887 

0.3798 

0 

0 

3031 

0.3798 

0 

0.01 

3183 

0.3798 

0 

0.02 

3342 

0.3799 

IE- 04 

0.02 

3509 

0.3799 

IE- 04 

0.03 

2612 

0.3797 

-lE-04 

-0.01 

2481 

0.3797 

-lE-04 

-0.02 

2357 

0.3797 

-lE-04 

-0.02 

2239 

0.3797 

-lE-04 

-0.03 

2127 

0.3797 

-2E-04 

-0.04 

Y(7) 

INITIAL 

MEAN  VALUE  =  252500 

CF  VALUE  =  0.3798164157 

Yj  +  %Yj 

CFV 

CHG.  CFV 

%CHG 

265125 

0.3821 

2 .3E-03 

0.62 

278381 

0.3846 

4 .8E-03 

1.27 

292300 

0.3872 

7.4E-03 

1.95 

306915 

0.3899 

0.0101 

2.67 

322261 

0.3928 

0.013 

3.43 

239875 

0.3774 

-2.4E-03 

-0.63 

227881 

0.3752 

-4 .6E-03 

-1.22 

216487 

0.373 

-6.8E-03 

-1.78 

205662 

0.371 

-8.8E-03 

-2.31 

195379 

0.3691 

-0.0107 

-2.81 

Y(8) 

INITIAL 

MEAN  VALUE  =  52500 

CF  VALUE  =  0.3798164157 

Yj  +  %Yj 

CFV 

CHG.  CFV 

asCHG 

55125 

0.3803 

4E-04 

0.12 

57881 

0.3808 

1 .OE-03 

0.26 

60775 

0.3813 

1.5E-03 

0.4 

63814 

0.3819 

2 .lE-03 

0.55 

67004 

0.3825 

2 .7E-03 

0.71 

49875 

0.3793 

-5E-04 

-0.13 

47381 

0.3788 

-1 .OE-03 

-0.26 

45012 

0.3784 

-1 .4E-03 

-0.37 

42761 

0.3779 

-1 .9E-03 

-0.48 

40623 

0 . 3775 

-2 .3E-03 

-0.59 
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*****  SENSITIVITY  ANALYSIS  ***** 
Y(9)  MEAN  VALUE  =  225000 

INITIAL  CF  VALUE  =  0.3798164157 


Yj  +  %Yj 

CFV 

CHG.  CFV 

%CHG 

236250 

0.3819 

2.1E-03 

0.55 

248062 

0.3841 

4.3E-03 

1.13 

260465 

0.3864 

6.6E-03 

1.74 

273488 

0.3888 

9.0E-03 

2.38 

287163 

0.3914 

0.0116 

3.05 

213750 

0 . 3777 

-2.2E-03 

-0.56 

203062 

0.3757 

-4 .lE-03 

-1.08 

192909 

0.3738 

-6.0E-03 

-1.58 

183263 

0.372 

-7.8E-03 

-2.06 

174100 

0.3703 

-9.6E-03 

-2.51 

YClO) 

INITIAL 

MEAN  VALUE  =  6000 

CF  VALUE  =  0.3798164157 

Yj  +  %Yj 

CFV 

CHG.  CFV 

%CHG 

6300 

0.3798 

0 

0.01 

6615 

0.3799 

IE- 04 

0.03 

6945 

0.3799 

IE- 04 

0.04 

7293 

0.38 

2E-04 

0.06 

7657 

0.3801 

3E-04 

0.08 

5700 

0.3797 

-lE-04 

-0.02 

5415 

0.3797 

-2E-04 

-0.03 

5144 

0.3796 

-2E-04 

-0.05 

4887 

0.3796 

-3E-04 

-0.06 

4642 

0.3795 

-3E-04 

-0.07 

Y<11)  MEAN  VALUE  =  1950 

INITIAL  CF  VALUE  =  0.3798164157 


Yj  +  %Y3 

CFV 

CHG.  CFV 

StCHG 

2047 

0.3798 

0 

0 

2149 

0.3798 

0 

0 

2257 

0.3798 

0 

0.01 

2370 

0.3798 

0 

0.02 

2488 

0.3799 

IE- 04 

0.02 

1852 

0.3797 

-lE-04 

-0.01 

1759 

0.3797 

-lE-04 

-0.01 

1671 

0.3797 

-lE-04 

-0.02 

1588 

0.3797 

-lE-04 

-0.02 

1508 

0.3797 

-lE-04 

-0.03 

*****  SENSITIVITY  ANALYSIS 
Y(12)  MEAN  VALUE  =  0.5 

INITIAL  CF  VALUE  =  0.3798164157 


Yj  +  %Yj 

CFV 

CHG.  CFV 

%CHG 

0.525 

0.3805 

7E-04 

0.19 

0.5512 

0.3813 

1 .5E-03 

0.39 

0.5787 

0.3821 

2.3E-03 

0.6 

0.6076 

0.3829 

3.1E-03 

0.83 

0.6379 

0.3838 

4 .OE-03 

1.06 

0.475 

0.379 

-8E-04 

-0.2 

0.4512 

0.3783 

-1 .5E-03 

-0.38 

0.4286 

0.3777 

-2 .lE-03 

-0.56 

0.4071 

0.377 

-2 .8E-03 

-0.72 

0.3867 

0.3764 

-3.4E-03 

-0.88 

Y(13)  MEAN  VALUE  =  0.5 

INITIAL  CF  VALUE  =  0.3798164157 


Yj  +  %Yj 

CFV 

CHG.  CFV 

%CHG 

0.525 

0.3805 

7E-04 

0.19 

0.5512 

0.3813 

1.5E-03 

0.39 

0.5787 

0.3821 

2 .3E-03 

0.6 

0.6076 

0.3829 

3.1E-03 

0.83 

0.6379 

0.3838 

4 .OE-03 

1.06 

0.475 

0.379 

-8E-04 

-0.2 

0.4512 

0.3783 

-1 .5E-03 

-0.38 

0.4286 

0 . 3777 

-2 .lE-03 

-0.56 

0.4071 

0.377 

-2 .8E-03 

-0.72 

0.3867 

0.3764 

-3.4E-03 

-0.88 

Y(14)  MEAN  VALUE  =0.5 

INITIAL  CF  VALUE  =  0.3798164157 


Yj  +  %Yj 

CFV 

CHG.  CFV 

SfiCHG 

0.525 

0.3805 

7E-04 

0.19 

0.5512 

0.3813 

1 .5E-03 

0.39 

0.5787 

0.3821 

2.3E-03 

0.6 

0.6076 

0.3829 

3.1E-03 

0.83 

0.6379 

0.3838 

4 .OE-03 

1.06 

0.475 

0.379 

-8E-04 

-0.2 

0.4512 

0.3783 

-1.5E-03 

-0.38 

0.4286 

0 . 3777 

-2 .lE-03 

-0.56 

0.4071 

0.377 

-2 .8E-03 

-0.72 

0.3867 

0.3764 

-3.4E-03 

-0.88 

101 


SENSITIVITY  ANALYSIS  *♦♦** 
Y(15)  MEAN  VALUE  =12,5 

INITIAL  CF  VALUE  =  0.3798164157 


Yj  +  %Y5 

CFV 

13.125 

0.3777 

13.7812 

0.3757 

14.4702 

0.3738 

15.1937 

0.372 

15.9533 

0.3702 

11.875 

0.3821 

11.2812 

0.3845 

10.7171 

0.3871 

10.1812 

0.3898 

9.6721 

0.3926 

CHG.  CFV 

%CHG 

-2.1E-03 

-0.56 

-4.1E-03 

-1.08 

-6.0E-03 

-1.58 

-7.9E-03 

-2.06 

-9.6E-03 

-2.51 

2 .3E-03 

0.6 

4 .7E-03 

1.25 

7 . 3E-03 

1.92 

0.01 

2.63 

0.0128 

3.38 

Y(16)  MEAN  VALUE  =  8560 

INITIAL  CF  VALUE  =  0.3798164157 


Yj  +  %Yj 

CFV 

CHG.  CFV 

%CHG 

8988 

0,3819 

2 .lE-03 

0.57 

94  37 

0.384 

4 .2E-03 

1.11 

9909 

0.386 

6.2E-03 

1.63 

104  04 

0.3879 

8.0E-03 

2.13 

10924 

0.3897 

9.8E-03 

2.6 

8132 

0.3774 

-2 .5E-03 

-0.64 

7725 

0.3748 

-5.0E-03 

-1.3 

7339 

0.3722 

-7.6E-03 

-2 

6972 

0.3694 

-0.0104 

-2.74 

6623 

0.3664 

-0.0134 

-3.52 

Y(17) 

INITIAL 

MEAN  VALUE  =  3.5 

CF  VALUE  =  0.3798164157 

Yj  +  SSYj 

CFV 

CHG.  CFV 

%CHG 

3.675 

0.3775 

-2.3E-03 

-0.61 

3.8587 

0.3751 

-4 .7E-03 

-1.24 

4.0516 

0.3726 

-7.2E-03 

-1.9 

4.2541 

0.3699 

-9.9E-03 

-2.6 

4.4668 

0.3672 

-0.0127 

-3.33 

3.325 

0.382 

2.2E-03 

0.6 

3.1587 

0.3842 

4  .4E-03 

1.17 

3.0007 

0.3863 

6.5E-03 

1.71 

2.8506 

0.3882 

8.4E-03 

2.23 

2.708 

0.3901 

0.0103 

2.72 
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Appendix  H;  Source  Code  for  Multiple  CFV  Comparisons 


10  REM  - 

20  REM  SOURCE  CODE,  VERSION  1.0 
30  REM  CRITERION  COMPARISON 
40  REM  FOR  GRADUATE  THESIS 
50  REM 

60  REM  WRITTEN  IN  TURBO-BASIC  XL  1.5 
70  REM  (C)  1985  BY  FRANK  OSTROWSKI 

80  REM  - 

90  REM 

100  DIM  Y(20)  ,OLDY(20)  .RESULT  (10 ,20)  , TOP  (5) 

110  FOR  Y1=0  TO  3 

120  FOR  Y2=l  TO  5 

130  FOR  Y3=0  TO  30  STEP  3 

140  FOR  Y4=0  TO  10 

150  FOR  Y5=12195  TO  13248  STEP  210.6 

160  FOR  Y6=500  TO  5000  STEP  450 

170  FOR  Y7=5000  TO  500000  STEP  49500 

180  FOR  Y8=5000  TO  100000  STEP  9500 

190  FOR  Y9=150000  TO  400000  STEP  25000 

200  FOR  Y10=2000  TO  10000  STEP  800 

210  FOR  Yll=500  TO  3400  STEP  290 

220  FOR  Y12=0  TO  1 

230  FOR  Y13=0  TO  1 

240  FOR  Y14=0  TO  1 

250  FOR  Y15=5  TO  20  STEP  2 

260  FOR  Y16=480  TO  16640  STEP  1616 

270  FOR  Y17=l  TO  6 

280  GOSUB  530 

290  IF  CF>TOP(l)  THEN  TOPd)  =CF;  V=1  :GOTO  570 

300  IF  CF>TOP(2)  THEN  TOP(2)  =CF:  V=2 -.GOTO  570 

310  IF  CF>TOP(3)  THEN  TOP (3) =CF : V=3 : GOTO  570 

320  IF  CF>TOP<4)  THEN  TOP(4) =CF: V=4 :GOTO  570 

330  IF  CF>TOP(5)  THEN  TOP(5)  =CF;V=5 -.GOTO  570 

340  REM  PLACE  HOLDER 

350  NEXT  Y17 

360  NEXT  Y16 

370  NEXT  Y15 

380  NEXT  Y14 

390  NEXT  Y13 

400  NEXT  Y12 

410  NEXT  Yll 

420  NEXT  YIO 

430  NEXT  Y9 

440  NEXT  Y8 

450  NEXT  Y7 

460  NEXT  Y6 
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470  NEXT  Y5 
480  NEXT  Y4 
490  NEXT  Y3 
500  NEXT  Y2 
510  NEXT  Y1 
520  GOTO  750 

530  CF1  =  0.24*  <<Y1* Y2)/15) +0.19* (((-0.5*  (Y3*Y4) ^2) +50) 
/-127058) 

540  CF2=CFl+0 . 16* ( ( (Y5+Y6+Y7+Y8+Y9+Y10+Y11) -175195) /856453) 
550  CF=CF2+0.22* (((Y12+Y13+Y14)/Y15)/0.6)+0.19* (((Y16/ 
(Y16+Y17)) -0.9979) /I .7E-03) 

560  RETURN 
570  RESULT (V,1)=Y1 
580  RESULT (V, 2) =Y2 
590  RESULT  (V,  3)  =Y3 
600  RESULT (V, 4) =Y4 
610  RESULT (V, 5) =Y5 
620  RESULT  (V,  6)  =Y6 
630  RESULT (V, 7) =Y7 
640  RESULT (V, 8) =Y8 
650  RESULT  (V,  9)  =Y9 
660  RESULT (V, 10) =Y10 
670  RESULT (V,11)=Y11 
680  RESULT (V, 12) =Y12 
690  RESULT(V,13)=Y13 
700  RESULT (V, 14) =Y14 
710  RESULT (V, 15) =Y15 
720  RESULT (V, 16) =Y16 
730  RESULT (V, 17) =Y17 
740  GOTO  340 

750  LPRINT  "CANDIDATE  SYSTEM  ANALYSIS  RESULTED  IN 
THE  FOLLOWING" 

760  LPRINT  "TOP  CHOICES" 

770  FOR  XX=1  TO  5 

780  LPRINT  "CHOICE  ";XX;"  CFV  =  ";TOP(XX) 

790  LPRINT  : LPRINT  "CANDIDATE  VALUES:" 

800  FOR  YY=1  TO  17 

810  LPRINT  "Y(";YY;")  =  " ;RESULT (XX , YY) 

820  NEXT  YY 
830  LPRINT  ; LPRINT 
840  NEXT  XX 
850  END 
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